How to solve the problem where YOLO model prediction results are null in Flutter

161 views Asked by At

I am developing an app to run keras and pytorch models on mobile. In the code below, the Keras model (classification) works well, but the output of the pytorch model (object detection, yolov5) is always null.

class _MyHomePageState extends State<PG_Main> {
  // Model and image related variables
  File? _originalImageFile;
  Interpreter? mobilenet;
  Model? yolo;
  img.Image? originalImage, outputImage;
  int? originalWidth, originalHeight;

  @override
  void initState() {
    super.initState();
    _loadModel();
  }

  Future<void> _loadModel() async {
    try {
      mobilenet = await Interpreter.fromAsset('external/model/lite_mbv2_model_opt_224.tflite');
      yolo = await PyTorchMobile.loadModel('external/model/best_9.torchscript.ptl');
      setState(() => _isModelLoaded = true);
    } catch (_) {
      print("Only supported for android and ios so far");
    }
  }

  Uint8List imageToByteList(img.Image image) {
    return Uint8List.fromList(img.encodePng(image));
  }

  img.Image resizeImage(img.Image originalImage, int width, int height) {
    return img.copyResize(originalImage, width: width, height: height);
  }

  Float32List normalizeImage(img.Image resizedImage) {
    // Extract RGB values and normalize them to the 0-1 range
    // ... (Normalization code)
  }

  Future<void> predictImage() async {
    // Run the MobileNet model on the image
    // ... (Prediction code)
  }

  img.Image drawBoundingBoxes(img.Image resizedImage, List<dynamic> outputTensor, double threshold) {
    // Draws bounding boxes around detected objects in the image
    // ... (BoundingBox code)
  }

  Future<void> runYolo() async {
    setState(() => _isLoading = true);
    
    if (yolo == null) {
      print('Error: YOLO model is not loaded.');
      return;
    }

    // Process image and get prediction
    List<double> flattenInput = input
          .expand((batch) =>
              batch.expand((y) => y.expand((x) => x).toList()).toList())
          .toList()
          .cast<double>();
      List<dynamic>? output;
      try {
        output = await yolo?.getPrediction(
            flattenInput, [1, 3, yoloHeight, yoloWidth], DType.float32);
        print(output);
        outputImage = drawBoundingBoxes(resizedImage, output!, 0.05);
        outputImage = resizeImage(outputImage!, originalWidth!, originalHeight!);

    setState(() => _isLoading = false);
  }
}

And these are the versions of the libraries I used.

dependencies:
  flutter:
    sdk: flutter
  auto_size_text: ^3.0.0
  permission_handler: ^11.0.1
  tflite_flutter: ^0.10.3
  pytorch_mobile: ^0.2.2
  image_picker: ^1.0.4
  image: ^3.0.1

I confirmed that there are no problems with the input, and there are no errors while the model is running, but only null is returned in any case. The return must be a tensor of shape [1,25200,9]. What is the problem?

0

There are 0 answers