Cloudformation template for creating an emr cluster with imdsv2

24 views Asked by At

I have two cloudformation templates: (1) - one for creating an ec2 instance with imdsv2 (2) - one for creating an emr cluster (imdsv1).

I am looking to update the template for the emr cluster so that its instances use imdsv2 instead of imdsv1. Normally when no information about imds is provided, then imdsv1 is used by default. However, I've been struggling to find the right place in (2) to add information related to imdsv2, and also what to add. I tried carrying the imdsv2 section from (1) to (2) and the validation failed during the creation.

Would appreciate if anyone could guide me through adding what's needed to make the instances of the emr cluster use imdsv2.

Below is the relevant section in (1) for creating a SINGLE ec2 instance with imdsv2: (the triple dots ... indicates other properties that are not relevant to imdsv2 so I skipped them for short)

Resources:
    IMDSv2LaunchTemplate:
      Type: AWS::EC2::LaunchTemplate
        Properties:
          LaunchTemplateName: IMDSV2
          LaunchTemplateData:
            MetadataOptions:
              HttpEndpoint: enabled
              HttpTokens: required
    my_instance:
      Type: AWS::EC2::Instance
      DependsOn: IMDSv2LaunchTemplate
      Properties: 
        LaunchTemplate:
          LaunchTemplateName: IMDSV2
          Version: 2
        InstanceType: t2.micro 

and below is the relevant section in (2)

Parameters:
Mappings:
Resources:
  EMRCluster:
    Type: AWS::EMR::Cluster
    Properties:
      Name: my-cluster
      Instances:
        MasterInstanceGroup:
          InstanceCount: 1
          InstanceType: m5.4xlarge
          Name: Master 
             ...
         CoreInstanceGroup:
             ...
   ...

Tried carrying that IMDSv2LaunchTemplate from (1) to Resources in (2), and added DependsOn: IMDSv2LaunchTemplate to (2) under Resources as well, but it says Template format error: Unresolved resource dependencies [IMDSv2LaunchTemplate,] in the Resources block of the template

Basically what I tried looked like the following:

Parameters:
Mappings:
Resources:
  IMDSv2LaunchTemplate:
    Type: AWS::EC2::LaunchTemplate
    Properties:
      LaunchTemplateName: IMDSV2
      LaunchTemplateData:
        MetadataOptions:
          HttpEndpoint: enabled
          HttpTokens: required
  EMRCluster:
    Type: AWS::EMR::Cluster
    DependsOn: IMDSv2LaunchTemplate
    Properties:
      Name: my-cluster
      Instances:
        MasterInstanceGroup:
          InstanceCount: 1
          LaunchTemplate:
            LaunchTemplateName: IMDSV2
            Version: 1
          InstanceType: m5.4xlarge
          Name: Master 
             ...
         CoreInstanceGroup:
             ...
   ...

The error I got is: Encountered unsupported property LaunchTemplate

1

There are 1 answers

0
D Malan On

You need to create a security configuration for your cluster by creating a AWS::EMR::SecurityConfiguration.

Something like this:

Parameters:
Mappings:
Resources:
  EMRCluster:
    Type: AWS::EMR::Cluster
    Properties:
      Name: my-cluster
      Instances:
        MasterInstanceGroup:
          InstanceCount: 1
          InstanceType: m5.4xlarge
          Name: Master 
             ...
         CoreInstanceGroup:
             ...
      SecurityConfiguration: !Ref securityConfiguration
  EMRSecurityConfiguration:
    Type: 'AWS::EMR::SecurityConfiguration'
    Properties:
      SecurityConfiguration:
      InstanceMetadataServiceConfiguration:
          MinimumInstanceMetadataServiceVersion: 2
          HttpPutResponseHopLimit: 1

Keep in mind that the documentation mentions that IMDSv1 will result in cluster failures for Amazon EMR 5.x or 6.x releases.