EmrServerlessCreateApplicationOperator networkConfiguration with multiple subnetIds

137 views Asked by At

If I pass more than one subnet Id to EmrServerlessCreateApplicationOperator via the networkConfiguration attribute, I receive an error.

If I use a single subnet Id the operator works fine. This is the network configuration and also shown is an extract of the operator:

network_config = {
    'subnetIds': [
        subnet_ids
    ],
    'securityGroupIds': [
        security_group_ids
    ]
}

EmrServerlessCreateApplicationOperator(
    task_id=task_identifier,
    job_type="SPARK",
    release_label="emr-6.6.0",
    config={"name":config_name, "networkConfiguration": network_config}
)

If I try passing more than one subnet Id (via the subnet_ids string value), whether comma separated either with or without quotes, I receive the following error from Airflow:

(An error occurred (ValidationException) when calling the CreateApplication operation: 1 validation error detected: Value at 'networkConfiguration.subnetIds' failed to satisfy constraint: Member must satisfy constraint: [Member must have length less than or equal to 32, Member must have length greater than or equal to 1, Member must satisfy regular expression pattern: [-0-9a-zA-Z]+.*]; 2522)

Does anyone have an example of how to correctly format multiple subnet Ids for the network configuration?

1

There are 1 answers

0
singleton On

As mentioned by @0x26res, if added as an additional item in the array rather than the string, this works as expected.