Synapse Analytics Auto ML Predict No module named 'azureml.automl'

Question

I follow the official tutotial from microsoft: https://learn.microsoft.com/en-us/azure/synapse-analytics/machine-learning/tutorial-score-model-predict-spark-pool

When I execute:

#Bind model within Spark session
 model = pcontext.bind_model(
     return_types=RETURN_TYPES, 
     runtime=RUNTIME, 
     model_alias="Sales", #This alias will be used in PREDICT call to refer  this   model
     model_uri=AML_MODEL_URI, #In case of AML, it will be AML_MODEL_URI
     aml_workspace=ws #This is only for AML. In case of ADLS, this parameter can be removed
 ).register()

I got : No module named 'azureml.automl'

My Notebook

Answer 1

I solved it. In my case it works best like this:

Imports

#Import libraries
from pyspark.sql.functions import col, pandas_udf,udf,lit
from notebookutils.mssparkutils import azureML
from azureml.core import Workspace, Model
from azureml.core.authentication import ServicePrincipalAuthentication
from azureml.core.model import Model
import joblib
import pandas as pd

ws = azureML.getWorkspace("AzureMLService")
spark.conf.set("spark.synapse.ml.predict.enabled","true")

Predict function

def forecastModel():
    model_path = Model.get_model_path(model_name="modelName", _workspace=ws)
    modeljob = joblib.load(model_path + "/model.pkl")

    validation_data = spark.read.format("csv") \
                            .option("header", True) \
                            .option("inferSchema",True) \
                            .option("sep", ";") \
                            .load("abfss://....csv")

    validation_data_pd = validation_data.toPandas()


    predict = modeljob.forecast(validation_data_pd)

    return predict

Answer 2

As per the repro from my end, the above code which you have shared works as excepted and I don't see any error message which you are experiencing.

I had even tested the same code on the newly created Apache spark 3.1 runtime and it works as expected.

I would request you to create a new cluster and see if you are able to run the above code.