How to prepare a Tensorflow model to be deployed in AI Platform

Question

I'm using a Tensorflow 2 pre-trained model (vgg16) to classify images. I have done the training on my local machine and now I wanted to deploy it GCloud AI Platform. This model's input is expecting an already decoded and resized image. However, when I try to call a predict, it returns an error that I'm exceeding the size limit. After looking into the documentation I saw that this method is very inefficient and so I must use base64 encoding to represent the image. The problem is that the model is not expecting an encoded image and I know that I need to modify the model before exporting it. However all the questions (1,2,3,4) that I see related to this are from TF version 1, using placeholders that are deprecated in TF 2.

Can someone help me on this?

Here is the code I'm using for transfer learning:

base_model = VGG16(input_shape=(IMG_SIZE, IMG_SIZE, 3), include_top=False, weights='imagenet')

inputs = tf.keras.Input(shape=IMG_SIZE + (3,))
x = base_model(inputs, training=False)
x = layers.GlobalAveragePooling2D()(x)
x = layers.Dropout(0.2)(x)
x = layers.Dense(128, activation="relu")(x)
x = layers.Dropout(0.2)(x)
x = layers.Dense(1, activation="sigmoid")(x)

model = Model(inputs, x)

model.compile(loss='binary_crossentropy',
              optimizer=Adam(lr=LEARN_RATE),
              metrics=metrics)

hist_log = model.fit(
    train_generator,
    steps_per_epoch=steps_train,
    epochs=EPOCHS,
    validation_data=valid_generator,
    validation_steps=steps_val,
    verbose=1,
    callbacks=callbacks)

model.save(export_path)

Answer 1

A single online prediction request must contain no more than 1.5 MB of data. Requests created using the gcloud tool can handle no more than 100 instances per file. We have seen this before, I would start with the following:

1. What's the size of your request and size of your response.
2. Have you tried custom prediction routines?
3. Have you tried casting output using uint8?