If I pass more than one subnet Id to EmrServerlessCreateApplicationOperator via the networkConfiguration attribute, I receive an error.
If I use a single subnet Id the operator works fine. This is the network configuration and also shown is an extract of the operator:
network_config = {
'subnetIds': [
subnet_ids
],
'securityGroupIds': [
security_group_ids
]
}
EmrServerlessCreateApplicationOperator(
task_id=task_identifier,
job_type="SPARK",
release_label="emr-6.6.0",
config={"name":config_name, "networkConfiguration": network_config}
)
If I try passing more than one subnet Id (via the subnet_ids string value), whether comma separated either with or without quotes, I receive the following error from Airflow:
(An error occurred (ValidationException) when calling the CreateApplication operation: 1 validation error detected: Value at 'networkConfiguration.subnetIds' failed to satisfy constraint: Member must satisfy constraint: [Member must have length less than or equal to 32, Member must have length greater than or equal to 1, Member must satisfy regular expression pattern: [-0-9a-zA-Z]+.*]; 2522)
Does anyone have an example of how to correctly format multiple subnet Ids for the network configuration?
As mentioned by @0x26res, if added as an additional item in the array rather than the string, this works as expected.