I am developing a CLI in swift and using FBSimulatorControl in it. I want to stream the data obtained from consumeData: delegate method. As per the data obtained in the method is data obtained from CVPixelBufferRef
I logged an issue in the github repo of FBSimulatorControl.
I tried to regenerate the CVPixelBuffer from the data and tried to obtain the CIImage from it and convert that to jpeg data. But it does not seem to work. Can anyone help me with this? Adding below the code I've tried
var pixelBuffer:CVPixelBuffer? = nil
let result = CVPixelBufferCreate(kCFAllocatorDefault, 750, 1334, kCVPixelFormatType_32BGRA, nil, &pixelBuffer)
if result != kCVReturnSuccess {
print("pixel buffer create success")
return
}
CVPixelBufferLockBaseAddress(pixelBuffer!, .init(rawValue: 0))
let yDestPlane:UnsafeMutableRawPointer? = CVPixelBufferGetBaseAddressOfPlane(pixelBuffer!, 0)
if yDestPlane == nil {
print("failed to create ydestplane")
return
}
let nsData = data as NSData
let rawPtr = nsData.bytes
memcpy(yDestPlane!, rawPtr, 750*1334*4)
CVPixelBufferUnlockBaseAddress(pixelBuffer!, .init(rawValue: 0))
kCVPixelFormatType_32BGRA, getbasea, 2208, NULL, NULL, nil, &pixelBuffer)
if #available(OSX 10.12, *) {
let ciImage = CIImage.init(cvPixelBuffer: pixelBuffer!)
let tempContext = CIContext.init(options: nil)
let videoImage = tempContext.createCGImage(ciImage, from: CGRect.init(x: 0, y: 0, width: CVPixelBufferGetWidth(pixelBuffer!), height: CVPixelBufferGetHeight(pixelBuffer!)))
let imageSize: NSSize = NSMakeSize(750, 1334)
let nsImageTest = NSImage(cgImage: videoImage!, size: imageSize)
if let bits = nsImageTest.representations.first as? NSBitmapImageRep {
let jpegFinalData = bits.representation(using: .JPEG, properties: [:])
if self.isStreaming {
var simIndex:Int?
for i in 0...(AUConnectionSimulatorMap.instance.simConnectionMap.count-1) {
if sim.udid == AUConnectionSimulatorMap.instance.simConnectionMap[i].sim.sim.udid {
simIndex = i
break
}
}
var finalData:Data = Data()
let finalDict = ["data":["type":"onScreenFrame","value":jpegFinalData!]] as Dictionary<String,Any>
try! finalData.pack(finalDict)
AUConnectionSimulatorMap.instance.simConnectionMap[simIndex!].ws?.write(data: finalData)
}
}
} else {
// Fallback on earlier versions
}