Is there a way to use Sagemaker Model Monitoring without enabling Data Capture for the model Endpoint on Sagemaker

Question

We are currently using Sagemaker to deploy the ML models but we do not want to enable data capture for Sagemaker to be able to use Model Data Monitoring due to PII constraints.

But we do have a way to keep a backup of all the inputs and outputs in a S3 location after PII cleanup. Is there a way to use these S3 locations directly to schedule the Model Data Monitoring jobs?

create_monitoring_schedule requires us to either pass the endpoint_input or the batch_transform_input

Followed these 2 notebooks : https://github.com/aws/amazon-sagemaker-examples/blob/main/sagemaker_model_monitor/introduction/SageMaker-ModelMonitoring.ipynb

https://github.com/aws/amazon-sagemaker-examples/blob/main/sagemaker_model_monitor/model_monitor_batch_transform/SageMaker-ModelMonitoring-Batch-Transform-Data-Quality-On-schedule.ipynb

We tried to POC whether we can pass the S3 location of our model endpoint inputs CSV to BatchTransformInput and do the data monitoring for realtime model in this way but that did not work.

mon_schedule_name = "DEMO-sagemaker-monitoring-poc-" + strftime(
    "%Y-%m-%d-%H-%M-%S", gmtime()
)

batch_transform_input=BatchTransformInput(
        data_captured_destination_s3_uri=<S3_location_of_model_input_CSV>,
        destination="/opt/ml/processing/input",
        dataset_format=MonitoringDatasetFormat.csv(header=False)
)

my_default_monitor.create_monitoring_schedule(
    monitor_schedule_name=mon_schedule_name,
    batch_transform_input=batch_transform_input,
    statistics=my_default_monitor.baseline_statistics(),
    constraints=my_default_monitor.suggested_constraints(),
    schedule_cron_expression=CronExpressionGenerator.hourly(),
    enable_cloudwatch_metrics=True,
)

This failed with following error

'MonitoringExecutionStatus': 'Failed', 'FailureReason': 'Job inputs had no data'

Answer 1

SageMaker Model Monitor makes use of Processing Jobs. Hence, you can kick of the Job yourself. Take a look at the Triggering execution manually section in this example on how to kick of the monitoring manually.