How to access all Images in the ImageReader Queue using Android Camera2 API

Question

In my project, I need to capture the frames of the camera streams continuously. Here is the current code snippet I used.

To set up the ImageReader, I set the maxImages to 20. Let is every time when callback is triggered, there would have 20 frames in the ImageReader Queue.

imageReader = ImageReader.newInstance(
            optimumSize.getWidth(),
            optimumSize.getHeight(),
            ImageFormat.YUV_420_888,
            20
);

Then to access each image of these 20 frames. I used the following snippet.

imageReader.setOnImageAvailableListener(new ImageReader.OnImageAvailableListener() {
    @Override
    public void onImageAvailable(ImageReader reader) {
        Image image = reader.acquireNextImage();
        while (image != null) {
            // some processing here.....
            image.close();
            image = reader.acquireNextImage();
        }
        if (image != null) {
            image.close();
        }
    }
}, processingHandler);

The key obstacle here is to be able to access each of 20 frames in a callback, for further image processing. However the aforementioned code seems have some problems (I can only access the latest image in the underlying queue). In fact, I only need to access a small patch (50 x 50 pixels) in each frames, specified by users.

The reason for doing this is that I need to get the 20 continuous frames data with sampling frequency being ~60Hz. This seems really hard to achieve if we can only access single frame in each callback, which can only achieve up to 30fps.

Any suggestions would be super welcome! Thanks!

Answer 1

Setting maxImages to 20 just means the queue will allow you to acquire 20 Images at the same time; it does not mean the onImageAvailable callback will fire only once 20 images are queued. The callback fires as soon as a single image is present.

Most camera devices run at 30fps max, so it's not surprising that's the speed you're seeing. Some cameras do have 60fps modes, but you have to explicitly switch to a CONTROL_AE_TARGET_FPS_RANGE of (60,60) to get that, and that's only if the device's CONTROL_AE_AVAILABLE_TARGET_FPS_RANGE values include that range.

60fps may also be resolution-limited (check the StreamConfigurationMap for minimum frame durations to find what resolutions can support 60fps, if you want to double-check).