I have an app that records video and overlays the preview with some data. The data can change while the video is being recorded and I would like to be able to have that data "baked in" to the video itself while the recording is taking place. Just like a watermark really.
I know it can be combined later using FFMPeg but that is rather cumbersome and would require that the user wait for that process to happen before they can review the video.
So far, I haven't found any way to do this on Android (iPhone yes..not that hard there).
I was thinking that it might be able to be done by piping the stream output of the Camera2 feed directly into FFMpeg, but that would bog down in devices of lesser power. Another option might be combining each frame output with an image generated in a background thread then using OpenGL (which I do not know) to combine them before the result gets recorded. ImageReader.aquireLatestImage() might work there I'm guessing.
From that android.media.image I'd convert to a bitmap like this:
val image: Image = reader.acquireLatestImage()
val buffer: ByteBuffer = image.getPlanes().get(0).getBuffer()
val bytes = ByteArray(buffer.capacity())
buffer.get(bytes)
val bitmapImage = BitmapFactory.decodeByteArray(bytes, 0, bytes.size, null)
Then I'd combine the image like this:
fun combineImages(frame: Bitmap?, image: Bitmap): Bitmap? {
var cs: Bitmap? = null
var rs: Bitmap? = null
rs = Bitmap.createScaledBitmap(
frame!!, image.width + 50,
image.height + 50, true
)
cs = Bitmap.createBitmap(
rs.width, rs.height,
Bitmap.Config.RGB_565
)
val comboImage = Canvas(cs)
comboImage.drawBitmap(image, 25, 25, null)
comboImage.drawBitmap(rs, 0, 0, null)
if (rs != null) {
rs.recycle()
rs = null
}
Runtime.getRuntime().gc()
return cs
}
Then covert the bitmap to an image (although I haven't found a way that can be done) Then inject that image into MediaRecorder.VideoSource.SURFACE (again, not sure how).
Is this plausible or is there a better way that's been done that I just haven't stumbled onto yet?
Thanks!