Camera2 how to detect focus on a metering area

Question

Afternoon,

I have an application that detects something in the scene. This runs on live camera preview. I now want to use the results of a (successful) detection to set certain points of interest for exposure and focus in the image.

I have implemented iOS based on FocusPointOfInterest and ExposurePointOfInterest (works fine). In Android i am using MeteringRectangle to set the ControlAfRegions and ControlAeRegions on a repeating capture request.

In iOS the autofocus is triggered automatically when the FocusPointOfInterest mentioned above loses focus on that area.

In Android the exposure reacts to different points (that change dynamically) but the focus doesn't. This also happens even in the simple case where one single area is defined for focus/exposure upon setup of the camera preview. In this case, I get exposure to react ok to the area defined but focus never changes even if the area loses focus. My understanding is that the focus metering area would drive new autofocus triggers if the area lost focus/changed significantly.

Tried

• Everytime a an area changes due to detection i call SetUpCaptureRequestMeteringAreas and TriggerAutoFocus but, because the area can change a lot due to a dynamic detection then the auto focus trigger enters an "infinite" loop
• Listen to the state of AF in the capture callback but after the first initial autofocus trigger the camera AF state goes from inactive->active scan->focus locked (which should be) but it just stays in focus locked no matter if the focus region is in focus or not
• Tried setting continuous picture for focus mode and still setting the focus metering area. The autofocus triggers sometimes but unfortunately this seems like based on the whole scene and ignores the metering area set.
• There is an interesting property that could be related but i couldn't manage any device that i possess to recognise this property or the capture control (control_af_scene_change) associated with it.

Apologies, code is Xamarin/.NET and not native but should be self explanatory.

Camera setup code

...
SetUpCaptureRequestBuilder(mPreviewRequestBuilder);
mPreviewRequest = mPreviewRequestBuilder.Build();
mPreviewSession.SetRepeatingRequest(mPreviewRequest, mCaptureCallback, mBackgroundHandler);

SetUpCaptureRequestMeteringAreas(new PointF(0.1f, 0.5f), new PointF(0.9f, 0.5f));
TriggerAutoFocus(null);
...

public void SetUpCaptureRequestMeteringAreas(PointF focusPt, PointF exposurePt)
{
    Rect sensorSize = (Rect)mCharacteristics.Get(CameraCharacteristics.SensorInfoActiveArraySize);

    // Create MeteringRectangle for focus
    MeteringRectangle focusAreaTouch = new MeteringRectangle(
                (int)(focusPt.X * sensorSize.Width()),
                (int)(focusPt.Y * sensorSize.Height()),
                (int)(sensorSize.Width() * 0.02),
                (int)(sensorSize.Height() * 0.02),
                MeteringRectangle.MeteringWeightMax);
    MeteringRectangle[] focusRegions = new MeteringRectangle[] { focusAreaTouch };

    // Create MeteringRectangle for exposure
    MeteringRectangle exposureAreaTouch = new MeteringRectangle(
                (int)(exposurePt.X * sensorSize.Width()),
                (int)(exposurePt.Y * sensorSize.Height()),
                (int)(sensorSize.Width() * 0.02),
                (int)(sensorSize.Height() * 0.02),
                MeteringRectangle.MeteringWeightMax);
    MeteringRectangle[] exposureRegions = new MeteringRectangle[] { exposureAreaTouch };

    // TODO check if supported first
    mPreviewRequestBuilder.Set(CaptureRequest.ControlAfRegions, focusRegions);
    mPreviewRequestBuilder.Set(CaptureRequest.ControlAeRegions, exposureRegions);
    mPreviewRequestBuilder.Set(CaptureRequest.ControlAwbRegions, exposureRegions);
    mPreviewRequest = mPreviewRequestBuilder.Build();
    try
    {
        mPreviewSession.SetRepeatingRequest(mPreviewRequest, null, mBackgroundHandler);
    }
    catch (CameraAccessException ex)
    {
        ex.PrintStackTrace();
    }
}

public void TriggerAutoFocus(CameraCaptureListener captureCallback)
{
    // autofocus trigger need to be submited in a single capture request
    mPreviewRequestBuilder.Set(CaptureRequest.ControlAfMode, new Java.Lang.Integer((int)ControlAFMode.Auto));
    mPreviewRequestBuilder.Set(CaptureRequest.ControlAfTrigger, new Java.Lang.Integer((int)ControlAFTrigger.Start));
    mPreviewRequest = mPreviewRequestBuilder.Build();
    
    try
    {
        mPreviewSession.Capture(mPreviewRequest, mCaptureCallback, mBackgroundHandler);
    }
    catch (CameraAccessException ex)
    {
        ex.PrintStackTrace();
    }
}