How to correctly resize detected face bound box match the camera preview

419 views Asked by At

I'm using ML kit face detection in my android application. I'm have implemented this with Camera 2 libaray. The functionality works fire and I'm getting a bound box for the face detected in the camera preview. But when I compare it with ML kit sample application. I think there is mistake in my bound box resize calculation.

private fun drawAdjustedFaceBoundBox(canvas: Canvas){

            val myPaint = Paint()
            myPaint.color = Color.rgb(220, 249, 10)
            myPaint.strokeWidth = 5f
            myPaint.style = Paint.Style.STROKE

            if(analyzedImageSize.width != 0 && analyzedImageSize.height != 0) {

                val horizontalScaleFactor = previewScreenSize.width / analyzedImageSize.width.toFloat()
                val verticalScaleFactor = previewScreenSize.height / analyzedImageSize.height.toFloat()

                val adjustedBoundRect = RectF()
                adjustedBoundRect.top = (objectBound.top * verticalScaleFactor) + surfaceTop.toFloat()
                adjustedBoundRect.left = objectBound.left * horizontalScaleFactor
                adjustedBoundRect.right = objectBound.right * horizontalScaleFactor
                adjustedBoundRect.bottom = (objectBound.bottom * verticalScaleFactor) + surfaceTop.toFloat()

                val adjustedMirrorObjectBound = RectF(adjustedBoundRect)

                if(mirrorCoordinates){
                    val originalRight = adjustedBoundRect.right
                    val originalLeft = adjustedBoundRect.left
                    //mirror the coordination since it's the front facing camera
                    adjustedMirrorObjectBound.left = (previewScreenSize.width - originalRight)
                    adjustedMirrorObjectBound.right = (previewScreenSize.width - originalLeft)
                }

                canvas.drawRect(adjustedMirrorObjectBound, myPaint)
            }
        }

This is the function I use for calculating resize bound box. I'll explain my approach.

ML kit analyse image size is 480 * 640. preview window of the camera is 1080 * 2131. And I'm using front facing camera. What I first did was get a scale factor for X and Y axis. X is named as horizontalScaleFactor Y is named as verticalScaleFactor. then I multiplied original bound box values with matching scale factors. surfaceTop.toFloat() is the value from the top of the screen to my camera preview view. In my example it's 0. Since I'm using front facing camera I mirrored the coordination after the scale up calculation. After applying these calculations there is abound box on my detected face but it's not covering my whole face like in the ML kit sample project. When I move face left and rignt my bound box move with the face correctly so seems like my bound box mirroring calculation is correct. I suspect something is wrong with my scale up calculation. Please refer to bellow image to get a idea about the difference between my application and sample application.

My Application preview bound box

My Application preview bound box

ML kit sample application bound box

ML kit sample application bound box

Basically my bound box doesn't cover two ears and chin areas. This seems like a more of a math problem rather than a code issue. Please help me identify what I'm doing wrong.

Edit

This is my repository I'm comparing against this sample repository face detection implementation. Please check the dev branch on my repo.

Edit 2

This is how it looks when application is running. I do not have access to the proxy image since it's not saving to the storage. Proxy image is created by the CameraX library which uses as for the image capturing.

0

There are 0 answers