Is it possible to pass params when using edge-deployed AutoML vision model?

Question

I have trained an AutoML Vision model using Google Cloud Platform. I trained an edge-specific version so I could deploy it in a docker image on my own hardware.

I followed the tutorial instructions here: https://cloud.google.com/vision/automl/docs/containers-gcs-tutorial

and have successfully performed some predictions using the example python code:

import base64
import io
import json

import requests


def container_predict(image_file_path, image_key, port_number=8501):
    """Sends a prediction request to TFServing docker container REST API.

    Args:
        image_file_path: Path to a local image for the prediction request.
        image_key: Your chosen string key to identify the given image.
        port_number: The port number on your device to accept REST API calls.
    Returns:
        The response of the prediction request.
    """

    with io.open(image_file_path, 'rb') as image_file:
        encoded_image = base64.b64encode(image_file.read()).decode('utf-8')

    # The example here only shows prediction with one image. You can extend it
    # to predict with a batch of images indicated by different keys, which can
    # make sure that the responses corresponding to the given image.
    instances = {
            'instances': [
                    {'image_bytes': {'b64': str(encoded_image)},
                     'key': image_key}
            ]
    }

    # This example shows sending requests in the same server that you start
    # docker containers. If you would like to send requests to other servers,
    # please change localhost to IP of other servers.
    url = 'http://localhost:{}/v1/models/default:predict'.format(port_number)

    response = requests.post(url, data=json.dumps(instances))
    print(response.json())

However, the response contains more predictions than I would like (40 even though I only want 5-10). I thought I would be able to add some parameters to the POST request to limit the number of predictions or perhaps filter based on the object detection score. Such functionality is outlined here: https://cloud.google.com/automl/docs/reference/rest/v1/projects.locations.models/predict#request-body

This documentation suggests that score_threshold or max_bounding_box_count should be able to be added to the request json package.

I tried something like this:

    instances = {
        'instances': [
            {'image_bytes': {'b64': str(encoded_image)},
            'key': key}
        ],
        'params': [
            {'max_bounding_box_count': 10}
        ]
    }

to no avail.

Is anyone aware of how to add params to the json request payload? or if the edge-deployed docker will even accept them?

Answer 1

You should try with something like:

{
   "instances":[
      {
         "image_bytes":{
            "b64":"/9j/4AAQSkZJRgABAQ....ABAAD2P//Z"
         },
         "params":{
            "maxBoundingBoxCount":"100"
         }
      }
   ]
}

This documentation shows an example.

Answer 2

Just wanted to know if it worked i am trying something similar with docker score_threshold while this doesn't give format error the response is still going over threshold

{
    "instances": [
        {
            "image_bytes": {
                "b64": "<base64 encoded image>"
            },
            "key": "your-chosen-image-key123"
        }
    ],
                "params": {
                "score_threshold": 0.7
            }
}