Failed to flush task queue within 120 seconds on Automatic ML

219 views Asked by At

I'm running a Automatic ML on Azure but it fails and this is the error the child jobs are giving me.

Starting the automl_batch_driver setup...
Set enable_streaming flag to False
Batch Run Id in the real script:  AutoML_11f87010-f7b0-4819-aecb-ad6b31f2469a_worker_11
2022-06-07 17:46:03,026817 - INFO - Beginning batch driver wrapper.
2022-06-07 17:46:03.297 - INFO - Successfully got the cache data store, caching enabled.
2022-06-07 17:46:03.297 - INFO - Took 0.13631367683410645 seconds to retrieve cache data store
2022-06-07 17:46:03.306 - INFO - No files are available for the cache store locally, downloading files from the Run.
2022-06-07 17:48:04.348 - ERROR - Type: {'code': 'ResourceExhausted', 'inner_error': {'code': 'Timeout'}}
Class: AzureMLException
Message: AzureMLException:
    Message: Failed to flush task queue within 120 seconds
    InnerException None
    ErrorResponse 
{
    "error": {
        "code": "UserError",
        "message": "Failed to flush task queue within 120 seconds",
        "inner_error": {
            "code": "ResourceExhausted",
            "inner_error": {
                "code": "Timeout"
            }
        }
    }
}
0

There are 0 answers