How to convert a Flutter CameraImage from a startImageStream() to PNG, JPG or Base64 String, REALLY FAST?

358 views Asked by At

I have been using the Camera package to create a real-time face detection through a cameraController.startImageStream(), so i get a CameraImage image per frame. I need to send that frame in PNG, JPG or Base64 String format via WebSockets. I have been using an adaptation from the top answer here, and it works (the image is still rotated 90°, but at least it does the conversion from CameraImage to PNG to Base64 String).

But the PNG conversion is REAL SLOW! :c

Sometimes, it takes 20 seconds per frame to process. Is there any other Faster solution?

The code that I am currently using is:

import 'dart:convert';
import 'package:camera/camera.dart';
mport 'package:image/image.dart' as imglib;

CameraImage to PNG method:

Future<List<int>> convertYUV420toImageColor(CameraImage image) async {
    try {
      final int width = image.width;
      final int height = image.height;
      final int uvRowStride = image.planes[1].bytesPerRow;
      final int? uvPixelStride = image.planes[1].bytesPerPixel;

      //print("uvRowStride: " + uvRowStride.toString());
      //print("uvPixelStride: " + uvPixelStride.toString());

      // imgLib -> Image package from https://pub.dartlang.org/packages/image
      var img =
          imglib.Image(width: width, height: height); // Create Image buffer

      // Fill image buffer with plane[0] from YUV420_888
      for (int x = 0; x < width; x++) {
        for (int y = 0; y < height; y++) {
          final int uvIndex =
              uvPixelStride! * (x / 2).floor() + uvRowStride * (y / 2).floor();
          final int index = y * width + x;

          final yp = image.planes[0].bytes[index];
          final up = image.planes[1].bytes[uvIndex];
          final vp = image.planes[2].bytes[uvIndex];
          // Calculate pixel color
          int r = (yp + vp * 1436 / 1024 - 179).round().clamp(0, 255);
          int g = (yp - up * 46549 / 131072 + 44 - vp * 93604 / 131072 + 91)
              .round()
              .clamp(0, 255);
          int b = (yp + up * 1814 / 1024 - 227).round().clamp(0, 255);
          // color: 0x FF  FF  FF  FF
          //           A   B   G   R
          img.data?.elementAt(index).setRgba(r, g, b, 255);
        }
      }

      imglib.PngEncoder pngEncoder = imglib.PngEncoder(level: 0);
      List<int> png = pngEncoder.encode(img);
      //muteYUVProcessing = false;
      return png;
    } catch (e) {
      print(">>>>>>>>>>>> ERROR:" + e.toString());
    }
    return [];
  }

PNG to Base64 method:

  Future<String> convertirImagenABase64AMAZON(List<int> bytes) async {
    // Encodes the bytes of a PNG to Base64 String
    final String base64String = base64.encode(bytes);
    return base64String;
  }
0

There are 0 answers