How to get private IPs of all instances in one remote-exec, terraform, aws

Question

I create with terraform 2 types of EC2 instances; the count of instances is always different. I want to get private IP's of all instances to one of the instances to a file.

For the moment I get IP's of the first instance created, but I don`t get the rest, no matter the count.

resource "aws_instance" "name1" {
  ...
  count         = 3
  ...
}

resource "aws_instance" "name2" {
  ...
  count         = 1
  ...
  provisioner "remote-exec" {
    inline = [
      ...
      "sudo echo '${aws_instance.name1[count.index].private_ip}' >> test.txt",
      "sudo echo '${aws_instance.name2[count.index].private_ip}' >> test2.txt",
    ]
  }
}

So test2.txt gets that 1 IP of that one instance created, which is fine. But the test.txt gets only IP of the first instance created of the three. I am trying different approaches of for_each but I think I am maybe approaching it in a wrong way.

Another approach was to save IP's locally with local-exec, but then I have to run remote script to get the files and I am running with problems with manual work (where I run terraform could change and therefore remote scp command have to be changed, so remote-exec would be cleaner).

Answer 1

I'm not sure what you seek is entirely possible in terraform. You will need some additional scripting on the instances to scan for the others, or somewhere. How could each instance know about the instances created after it?

However, if you are okay with haveing all the IP addresses of the name1 instances and the IP address for the current name2 instance running the provisioner, that should be possible with something like:

resource "aws_instance" "name1" {
  count = 3
}

resource "aws_instance" "name2" {
  count = 1

  provisioner "remote-exec" {
    inline = [
      for ip in concat(aws_instance.name1[*].private_ip, [self.private_ip])
      : "echo ${ip} >> ips.txt"
    ]
  }
}

But I haven't tested that to be sure. It nevertheless, does not suffer from circular dependencies. I'd highly recommend just getting the IP addresses via an output though, then pushing that to an SSM parameter or S3 blob. You can then have cloud-init user data pull those values to write the hosts file.

You can get all the IP addresses like:

resource "aws_instance" "name1" {
  count = 3
}

resource "aws_instance" "name2" {
  count = 1
}

output "ips" {
  value = concat(aws_instance.name1[*].private_ip, aws_instance.name2[*].private_ip)
}

In lieu of this output, you could use this same list to store the values somewhere accessible to the instances when running cloud-init.

---

An example of what I mean, would be something like:

resource "aws_instance" "name1" {
  count = 3

  user_data = base64encode(templatefile("${path.module}/cloud-init.sh", {
    SSM_PATH = "ec2/ips"
  }))
}

resource "aws_instance" "name2" {
  count = 1

  user_data = base64encode(templatefile("${path.module}/cloud-init.sh", {
    SSM_PATH = "ec2/ips"
  }))
}

resource "aws_ssm_parameter" "ips" {
  name  = "ec2/ips"
  type  = "String"
  value = concat(aws_instance.name1[*].private_ip, aws_instance.name2[*].private_ip)
}

where cloud-init.sh is:

#!/usr/bin/env sh
sleep 30
# Better to wait until the parameter is present.
# But this is a bad way to do it.
aws ssm get-parameter --name "${SSM_PATH}" | jq -r .Parameter.Value
echo "some loop that writes your /etc/hosts file"

But I haven't verified this, so your mileage may vary.