How to remove background from an image using TensorFlow Lite?

Question

I am creating an Android App in Kotlin in which I want to remove background from a person's portrait image in real time.

(This code is meant to be imbedded in a video calling app whose one of the feature is to remove person's background during video calls for privacy issues.)

I have downloaded the starter app of TensorFlow Lite from here. It is generating a mask and an overlay of mask and the captured image. How can we use that mask to take a cut-out of the person and replace background with any image from the gallery.
I haven't done any work with TensorFlow Lite in the past, so any help would be highly appreciated.

Thanks in advance.

Answer 1

The project of TF Lite works fine and you get the generated mask from input image... Then you have to use something different to achieve the desired result. Just check out PorterDuff.Mode!

I have created for you a helper function to use it and get the desired bitmap. From there you can continue and load it inside an ImageView:

fun cropBitmapWithMask(original: Bitmap, mask: Bitmap?): Bitmap? {
        if (mask == null
        ) {
            return null
        }
        val w = original.width
        val h = original.height
        if (w <= 0 || h <= 0) {
            return null
        }
        val styled = Bitmap.createBitmap(w, h, Bitmap.Config.ARGB_8888)
        val canvas = Canvas(styled)
        val paint =
            Paint(Paint.ANTI_ALIAS_FLAG)
        paint.xfermode = PorterDuffXfermode(PorterDuff.Mode.DST_IN)
        canvas.drawBitmap(original, 0f, 0f, null)
        canvas.drawBitmap(mask, 0f, 0f, paint)
        paint.xfermode = null
        return styled
    }

Tag me if you have more questions. 

Cheers