Google Mobile Vision API not detecting text in portrait mode

Question

I have an activity which uses the Fotoapparat Library to obtain a picture and save it to a file. Next I use Google Mobile Vision API to create a bitmap of that file and detect text. I have used the standard code provided for this.

    TextRecognizer ocrFrame = new TextRecognizer.Builder(context).build();
    Frame frame = new Frame.Builder().setBitmap(BitmapFactory.decodeFile(pathToPhoto)).build();

    SparseArray<TextBlock> sparseTextBlocks = ocrFrame.detect(frame);
    if (sparseTextBlocks.size() <= 0)
        return null;

    ArrayList<TextBlock> textBlocks = new ArrayList<>();
    for (int i = 0; i < sparseTextBlocks.size(); i++) {
        textBlocks.add(sparseTextBlocks.get(sparseTextBlocks.keyAt(i)));
    }

The OCR works perfectly in landscape mode but in portrait mode it hardly detects any text. I have verified by displaying the image that the image is not inverted in portrait mode. It gives a vertical image. I really can not figure out why this is happening. any clues?

Answer 1

Here is another alternative for implementing the Mobile Vision API

// imageBitmap is the Bitmap image you're trying to process for text
if(imageBitmap != null) {

    TextRecognizer textRecognizer = new TextRecognizer.Builder(this).build();

    if(!textRecognizer.isOperational()) {
        // Note: The first time that an app using a Vision API is installed on a
        // device, GMS will download a native libraries to the device in order to do detection.
        // Usually this completes before the app is run for the first time.  But if that
        // download has not yet completed, then the above call will not detect any text,
        // barcodes, or faces.
        // isOperational() can be used to check if the required native libraries are currently
        // available.  The detectors will automatically become operational once the library
        // downloads complete on device.
        Log.w(LOG_TAG, "Detector dependencies are not yet available.");

        // Check for low storage.  If there is low storage, the native library will not be
        // downloaded, so detection will not become operational.
        IntentFilter lowstorageFilter = new IntentFilter(Intent.ACTION_DEVICE_STORAGE_LOW);
        boolean hasLowStorage = registerReceiver(null, lowstorageFilter) != null;

        if (hasLowStorage) {
            Toast.makeText(this,"Low Storage", Toast.LENGTH_LONG).show();
            Log.w(LOG_TAG, "Low Storage");
        }
    }


    Frame imageFrame = new Frame.Builder()
            .setBitmap(imageBitmap)
            .build();

    SparseArray<TextBlock> textBlocks = textRecognizer.detect(imageFrame);

    for (int i = 0; i < textBlocks.size(); i++) {
        TextBlock textBlock = textBlocks.get(textBlocks.keyAt(i));

        Log.i(LOG_TAG, textBlock.getValue()); 
        // Do something with value
    }
}

You need to ensure you include the mobile vision dependency in the module's build.gradle

dependencies {
    compile 'com.google.android.gms:play-services-vision:9.4.0'
} 

And also include this on the app's Android Manifest

<meta-data
    android:name="com.google.android.gms.vision.DEPENDENCIES"
    android:value="ocr" />

Overall your code looks fine, i think it could be the way your library saves the orientation of the pictures could conflicting with the Mobile Vision API, try using native android captures on a side project or another library, if your app still not working try saving the still in landscape even if they are taken on portrait that could help as well

Hope it helps