How to fix: "error": "Prediction failed: unknown error." in custom prediction routine with scikit-learn?

330 views Asked by At

I am trying to write a custom prediction routine on Google's AI Platform using scikit-learn's MLPClassifier. I have packaged and deployed the model successfully, but when I request online predictions via gcloud ai-platform predict, the error I get the error "error": "Prediction failed: unknown error." I then went to the console to test my model manually in the "Test & Use" section of my model and received the same error.

The training vectors are numpy arrays with 6 elements (e.g. [1,2,3,4,5,6]) and the targets are either 0, 1, or 2.

Here is my preprocess.py code:

import numpy as np
class MySimpleScaler(object):
  def __init__(self):
    self._means = None
    self._stds = None

  def preprocess(self, data):
    if self._means is None: # during training only
      self._means = np.mean(data, axis=0)

    if self._stds is None: # during training only
      self._stds = np.std(data, axis=0)
      if not self._stds.all():
        raise ValueError('At least one column has standard deviation of 0.')

    return (data - self._means) / self._stds

Here is my predictor.py code:

import os
import pickle
import numpy as np
from sklearn.externals import joblib
from sklearn.neural_network import MLPClassifier

class MyPredictor(object):
  def __init__(self, model, preprocessor):
    self._model = model
    self._preprocessor = preprocessor
    self._class_names = ["0-6 months", "7-18 months", "18+ months"]

  def predict(self, instances, **kwargs):
    inputs = np.asarray(instances)
    preprocessed_inputs = self._preprocessor.preprocess(inputs)
    if kwargs.get('probabilities'):
      probabilities = self._model.predict_proba(preprocessed_inputs)
      return probabilities.tolist()
    else:
      outputs = self._model.predict(preprocessed_inputs)
      return [self._class_names[class_num] for class_num in outputs]

  @classmethod
  def from_path(cls, model_dir):
    model_path = os.path.join(model_dir, 'model.joblib')
    model = joblib.load(model_path)

    preprocessor_path = os.path.join(model_dir, 'preprocessor.pkl')
    with open(preprocessor_path, 'rb') as f:
      preprocessor = pickle.load(f)

    return cls(model, preprocessor)

Here is the code where I train and export my model:

scaler = MySimpleScaler()
y = data[:, [0]]
features_scaled = scaler.preprocess(data[:, 1:])
scaled_data = np.concatenate((y, features_scaled), 1) # put the scaled features and the y column back 
    together
X = scaled_data[:, 1:]
clf = MLPClassifier()
clf.fit(X, y)

# export the model
joblib.dump(clf, 'model.joblib')
with open ('preprocessor.pkl', 'wb') as f:
    pickle.dump(scaler, f)

setup.py:

from setuptools import setup

setup(
    name='my_custom_code',
    version='0.1',
    include_package_data=True,
    scripts=['predictor.py', 'preprocess.py'])

I have tried serving online predictions with the input.json file looking like this

[1,2,3,4,5,6]

with this command

gcloud ai-platform predict --version $CORRECT_VERSION --model $CORRECT_MODEL --json-instances 
   input.json

and I get the error above. Can someone please help? I wish Google AI Platform had more informative error messages.

0

There are 0 answers