incorrect ephemeral data from meta-data/block-device-mapping in ec2

Question

we're using Ansible to provision ec2 instances, deploy our application on them and then create an AMI based on that instance, update the launch config so the autoscaling group always launch a new version of the app.

we re-use much of the ansible's playbooks for different Apps (diff instance sizes, families, etc). one of the tasks that this runs is to check the instance meta-data to find out if there are any ephemeral devices present, and if so, mount them and persist them in the /etc/fstab and in the cloud-config.

we run into an issue with the instance meta-data similar to what was described 3 years ago in this thread: https://forums.aws.amazon.com/thread.jspa?messageID=489889

right now we're doing some testing with t2.large instance, as you know, this instances are ebs only, but when we curl the instance metadata we get that there are 2 ephemeral disks presents: ubuntu@ip-xxx-xxx-xxx-xxx:~$ curl http://169.254.169.254/latest/meta-data/block-device-mapping/ ami ebs1 ebs2 ephemeral0 ephemeral1

the thing is that those ephemeral devices don't actually exists, so when our script tries to persist them in cloud-config, cloud-config ends up as a broken yaml blob....this is a major issue, because it causes the user_data to fail when an instance is launching.

any ideas anyone?

Answer 1

If this is a known issue that amazon team is not going to fix, you can workaround it.

Quick script to check actual existence:

#!/bin/bash

for MAPID in $(curl -s http://169.254.169.254/latest/meta-data/block-device-mapping/); do
  BLKDEV=$(curl -s http://169.254.169.254/latest/meta-data/block-device-mapping/$MAPID/ | sed 's#^/dev/##' | sed 's/^sd/xvd/')
  if blkid | grep -q $BLKDEV; then
    echo $BLKDEV present
  else
    echo $BLKDEV not present
  fi
done

Checked it with c1.medium and t2.medium. t2 output:

$ curl -s http://169.254.169.254/latest/meta-data/block-device-mapping/
ami
ebs1
ephemeral0
ephemeral1
root

$ ./test.sh
xvda1 present
xvdd present
xvdb not present
xvdc not present
xvda1 present