Bridging Android CameraX into React Native

Question

I am creating a native UI component for CameraX to React Native. First tried triggering in activity to check and worked fine. Looking out to extract as UI View for React Native. Its just showing a blank white screen.

• camera preview is not showing (100% w and h for view styles are defined in JS)?

//bind to lifecycle:
CameraX.bindToLifecycle((AppCompatActivity) mContext.getCurrentActivity(), preview);

My snippet below:

camera_layout:

<androidx.constraintlayout.widget.ConstraintLayout ...>
    <TextureView android:id="@+id/textureView" ... />
</androidx.constraintlayout.widget.ConstraintLayout>

CameraXView:

public class CameraXView extends View {
    private static final String TAG =  ReactContextBaseJavaModule.class.getSimpleName();
    TextureView textureView;
    Preview preview;
    ThemedReactContext mContext;

    public CameraXView(ThemedReactContext context) {
        super(context);
        mContext = context;
        ConstraintLayout layout = (ConstraintLayout) LayoutInflater.from(context).inflate(R.layout.camera_layout, null);
        textureView = layout.findViewById(R.id.textureView);
        startCamera();
    }
}

Start CameraX functionalities:

public void startCamera() {
        CameraX.unbindAll();
        
        Rational aspectRatio = new Rational(textureView.getWidth(), textureView.getHeight());
        Size screen = new Size(textureView.getWidth(), textureView.getHeight()); //size of the screen

        PreviewConfig pConfig = new PreviewConfig.Builder().setTargetAspectRatio(aspectRatio).setTargetResolution(screen).build();
        Preview preview = new Preview(pConfig);

        preview.setOnPreviewOutputUpdateListener(
                new Preview.OnPreviewOutputUpdateListener() {
                    @Override
                    public void onUpdated(Preview.PreviewOutput output) {
                        ViewGroup parent = (ViewGroup) textureView.getParent();
                        parent.removeView(textureView);
                        parent.addView(textureView, 0);

                        textureView.setSurfaceTexture(output.getSurfaceTexture());
                        updateTransform();
                    }
                });

        //bind to lifecycle:
        CameraX.bindToLifecycle((AppCompatActivity) mContext.getCurrentActivity(), preview);
    }

ReactViewManager: Passing current activity from reactContext to view instance

public class ReactCameraXManager extends SimpleViewManager<CameraXView> implements LifecycleEventListener {
   public static final String TAG = ReactContextBaseJavaModule.class.getSimpleName();
   public static final String REACT_CLASS = "CameraXView";
   private Activity mActivity;

   public ReactCameraXManager(ReactApplicationContext reactContext) {
       super();
       mActivity = reactContext.getCurrentActivity();
       reactContext.addLifecycleEventListener(this);
   }

   @Override
   public String getName() {
       return REACT_CLASS;
   }

   @Override
   public CameraXView createViewInstance(final ThemedReactContext context) {
       return new CameraXView(context);
   }

Actual Log after camera view rendered:

09-26 20:15:18.994 8156-8178/com.app D/Camera: Use cases [Preview:androidx.camera.core.Preview-e0193733-9ece-4c39-85e5-b93a6f2bf2e1] OFFLINE for camera 0
09-26 20:15:18.994 8156-8178/com.app D/Camera: Closing camera: 0
09-26 20:15:18.995 8156-8178/com.app I/RequestQueue: Repeating capture request cancelled.
09-26 20:15:19.174 8156-8168/com.app E/BufferQueueProducer: [SurfaceTexture-1-8156-4] cancelBuffer: BufferQueue has been abandoned
09-26 20:15:19.174 8156-8169/com.app E/BufferQueueProducer: [SurfaceTexture-1-8156-4] cancelBuffer: BufferQueue has been abandoned
09-26 20:15:19.174 8156-8191/com.app E/BufferQueueProducer: [SurfaceTexture-1-8156-4] cancelBuffer: BufferQueue has been abandoned
09-26 20:15:19.174 8156-8168/com.app E/BufferQueueProducer: [SurfaceTexture-1-8156-4] cancelBuffer: BufferQueue has been abandoned
09-26 20:15:19.175 8156-8169/com.app E/BufferQueueProducer: [SurfaceTexture-1-8156-4] cancelBuffer: BufferQueue has been abandoned
09-26 20:15:19.175 8156-8191/com.app E/BufferQueueProducer: [SurfaceTexture-1-8156-4] cancelBuffer: BufferQueue has been abandoned
09-26 20:15:19.208 8156-8178/com.app D/Camera: Closing Capture Session
09-26 20:15:19.209 8156-8178/com.app D/Camera: CameraDevice.onClosed(): 0
09-26 20:15:19.209 8156-8178/com.app D/Camera: Closing Capture Session
09-26 20:15:19.216 8156-8178/com.app D/Camera: Use cases [Preview:androidx.camera.core.Preview-a29229b4-7347-4920-8407-d60e97ec229b] ONLINE for camera 0
09-26 20:15:19.216 8156-8178/com.app D/Camera: Opening camera: 0
09-26 20:15:19.216 8156-8178/com.app D/UseCaseAttachState: All use case: [androidx.camera.core.Preview-a29229b4-7347-4920-8407-d60e97ec229b] for camera: 0
09-26 20:15:19.230 8156-8178/com.app I/CameraManager: Using legacy camera HAL.
09-26 20:15:19.432 8156-8178/com.app D/UseCaseAttachState: Active and online use case: [] for camera: 0
09-26 20:15:19.432 8156-8178/com.app D/UseCaseAttachState: All use case: [androidx.camera.core.Preview-a29229b4-7347-4920-8407-d60e97ec229b] for camera: 0
09-26 20:15:19.432 8156-8178/com.app D/Camera: Closing Capture Session
09-26 20:15:19.432 8156-8178/com.app D/Camera: CameraDevice is null
09-26 20:15:19.432 8156-8178/com.app D/Camera: Use case Preview:androidx.camera.core.Preview-a29229b4-7347-4920-8407-d60e97ec229b ACTIVE for camera 0
09-26 20:15:19.432 8156-8178/com.app D/UseCaseAttachState: Active and online use case: [androidx.camera.core.Preview-a29229b4-7347-4920-8407-d60e97ec229b] for camera: 0
09-26 20:15:19.432 8156-8178/com.app D/Camera: CameraDevice.onOpened(): 0
09-26 20:15:19.432 8156-8178/com.app D/UseCaseAttachState: All use case: [androidx.camera.core.Preview-a29229b4-7347-4920-8407-d60e97ec229b] for camera: 0
09-26 20:15:19.432 8156-8178/com.app D/Camera: Closing Capture Session
09-26 20:15:19.433 8156-8178/com.app D/CaptureSession: Opening capture session.
09-26 20:15:19.435 8156-8178/com.app I/CameraDeviceState: Legacy camera service transitioning to state CONFIGURING
09-26 20:15:19.437 8156-9611/com.app I/RequestThread-0: Configure outputs: 1 surfaces configured.
09-26 20:15:19.437 8156-9611/com.app D/Camera: app passed NULL surface
09-26 20:15:19.465 8156-8178/com.app I/CameraDeviceState: Legacy camera service transitioning to state IDLE
09-26 20:15:19.466 8156-8178/com.app D/CaptureSession: Attempting to send capture request onConfigured
09-26 20:15:19.466 8156-8178/com.app D/CaptureSession: Issuing request for session.
09-26 20:15:19.468 8156-8178/com.app I/RequestQueue: Repeating capture request set.
09-26 20:15:19.468 8156-8178/com.app D/CaptureSession: CameraCaptureSession.onConfigured()
09-26 20:15:19.468 8156-8178/com.app D/CaptureSession: CameraCaptureSession.onReady()
09-26 20:15:19.468 8156-8178/com.app D/CaptureSession: CameraCaptureSession.onReady()
09-26 20:15:19.471 8156-9611/com.app W/LegacyRequestMapper: convertRequestMetadata - control.awbRegions setting is not supported, ignoring value
09-26 20:15:19.471 8156-9611/com.app W/LegacyRequestMapper: Only received metering rectangles with weight 0.
09-26 20:15:19.471 8156-9611/com.app W/LegacyRequestMapper: Only received metering rectangles with weight 0.
09-26 20:15:19.675 8156-9612/com.app I/CameraDeviceState: Legacy camera service transitioning to state CAPTURING
09-26 20:18:22.352 8156-8167/com.app I/art: Background partial concurrent mark sweep GC freed 483777(15MB) AllocSpace objects, 0(0B) LOS objects, 35% free, 29MB/45MB, paused 1.554ms total 104.490ms
09-26 20:19:30.100 8156-8167/com.app I/art: Background sticky concurrent mark sweep GC freed 473262(15MB) AllocSpace objects, 0(0B) LOS objects, 34% free, 29MB/45MB, paused 771us total 166.198ms
09-26 20:20:35.488 8156-8167/com.app I/art: Background partial concurrent mark sweep GC freed 491511(15MB) AllocSpace objects, 0(0B) LOS objects, 35% free, 29MB/45MB, paused 968us total 100.951ms

Note: I don't see any react-native package for this. So planned to create a new package.

Answer 1

I found that PreviewView is not added in the parent (even getChildCount was returning 0)

It worked after attaching the layout with root:

FrameLayout layout = (FrameLayout) LayoutInflater.from(context).inflate(R.layout.camera_layout, this, true);

instead of this:

FrameLayout layout = (FrameLayout) LayoutInflater.from(context).inflate(R.layout.camera_layout, null);

Note: Performance is not that great. FPS is too low on most of the devices.