gcloud jobs submit prediction 'can't decode json' with --data-format=TF_RECORD

Question

I pushed up some test data to gcloud for prediction as a binary tfrecord-file. Running my script I got the error ('No JSON object could be decoded', 162). What do you think I am doing wrong?

To push a prediction job to gcloud, i use this script:

REGION=us-east1
MODEL_NAME=mymodel
VERSION=v_hopt_22
INPUT_PATH=gs://mydb/test-data.tfr
OUTPUT_PATH=gs://mydb/prediction.tfr
JOB_NAME=pred_${MODEL_NAME}_${VERSION}_b

args=" --model "$MODEL_NAME
args+=" --version "$VERSION

args+=" --data-format=TF_RECORD"
args+=" --input-paths "$INPUT_PATH
args+=" --output-path "$OUTPUT_PATH

args+=" --region "$REGION

gcloud ml-engine jobs submit prediction $JOB_NAME $args

test-data.tfr has been generated from a numpy array, as so:

import numpy as np

filename = './Datasets/test-data.npz'
data = np.load(filename)
features = data['X'] # features[channel, example, feature]
np_features = np.swapaxes(features, 0, 1) # features[example, channel, feature]

import tensorflow as tf
import nnscoring.data as D

def floats_feature(arr):
    return tf.train.Feature(float_list=tf.train.FloatList(value=arr.flatten().tolist()))

writer = tf.python_io.TFRecordWriter("./Datasets/test-data.tfr")

for i, np_example in enumerate(np_features):
    if i%1000==0: print(i)
    tf_feature = {  
        ch: floats_feature(x)
        for ch, x in zip(D.channels, np_example)
    }
    tf_features = tf.train.Features(feature=tf_feature)
    tf_example = tf.train.Example(features=tf_features)
    writer.write(tf_example.SerializeToString())

writer.close()

Update (following yxshi):

I define the following serving function

def tfrecord_serving_input_fn():
    import tensorflow as tf
    seq_length = int(dt*sr) 
    examples = tf.placeholder(tf.string, shape=())
    feat_map = {
        channel: tf.FixedLenSequenceFeature(shape=(seq_length,),
            dtype=tf.float32, allow_missing=True)
        for channel in channels
    }
    parsed = tf.parse_single_example(examples, features=feat_map)
    features = {
        channel: tf.expand_dims(tensor, -1)
        for channel, tensor in parsed.iteritems()
    }
    from collections import namedtuple
    InputFnOps = namedtuple("InputFnOps", "features labels receiver_tensors")
    tf.contrib.learn.utils.input_fn_utils.InputFnOps = InputFnOps
    return InputFnOps(features=features, labels=None, receiver_tensors=examples)
    # InputFnOps = tf.contrib.learn.utils.input_fn_utils.InputFnOps
    # return InputFnOps(features, None, parsed)
    # Error: InputFnOps has no attribute receiver_tensors

.., which I pass to generate_experiment_fn as so:

export_strategies = [
      saved_model_export_utils.make_export_strategy(
          tfrecord_serving_input_fn,
          exports_to_keep = 1,
          default_output_alternative_key = None,
  )]

  gen_exp_fn = generate_experiment_fn(
      train_steps_per_iteration = args.train_steps_per_iteration,
      train_steps        = args.train_steps,
      export_strategies  = export_strategies
  )

(aside: note the dirty patch of InputFnOps)

Answer 1

It looks like the input is not correctly specified in the inference graph. To use tf_record as input data format, your inference graph must accept strings as the input placeholder. In your case, you should have something like below in your inference code:

 examples = tf.placeholder(tf.string, name='input', shape=(None,))
 with tf.name_scope('inputs'):
   feature_map = {
     ch: floats_feature(x)
     for ch, x in zip(D.channels, np_example)
   }
   parsed = tf.parse_example(examples, features=feature_map)
   f1 = parsed['feature_name_1']
   f2 = parsed['feature_name_2']

 ...

A close example is here: https://github.com/GoogleCloudPlatform/cloudml-samples/blob/master/flowers/trainer/model.py#L253

Hope it helps.