GCP ML Tensorflow serving with authorization for GRPC

230 views Asked by At

I needed help in setting up GRPC for an ML model deployed on GCP. I have deployed the model and want to serve it using GRPC but not able to find any documentation on how to do that. Does anyone have any info on the below:-

  1. How to activate GRPC for TF serving model deployed at ml.googleapis.com?
  2. What are the steps that are needed for the model to be served using GRPC with authorization?

Currently there is sufficient documentation for serving with JSON requests but these requests are becoming very big and I need an alternative for reducing size and thats why needed to find more doc on it. right now I am getting error

InactiveRpcError of RPC that terminated with:
    status = StatusCode.UNKNOWN
    details = "Stream removed"

let me know if you would need more info.

this the code i am using

from google import auth as google_auth
from google.auth.transport import requests as google_auth_transport_requests
from google.auth.transport import grpc as google_auth_transport_grpc
from tensorflow_serving.apis import predict_pb2

def _make_grpc_request(examples):
    scoped_credentials, _ = google_auth.default(scopes=('https://www.googleapis.com/auth/cloud-platform',))
    request = google_auth_transport_requests.Request()
    channel = google_auth_transport_grpc.secure_authorized_channel(
    scoped_credentials, request, 'ml.googleapis.com:443')

    """Builds and sends request to TensorFlow model server."""
    request = predict_pb2.PredictRequest()
    request.model_spec.name = 'nudity2'                           
    request.model_spec.signature_name = 'serving_default'
    request.inputs["input"].CopyFrom(
        tf.make_tensor_proto(examples))
    p_s=PredictionServiceStub(channel)
    response = p_s.Predict(request, 1000)
    response
    return response

    return _make_grpc_request
0

There are 0 answers