Can I use one packer builder with many provisioners and still run parallel builds?

Question

I have what seems to be like a valid use case for an unsupported - afaik - scenario, using packer.io and I'm worried I might be missing something...

So, in packer, I can add:

• many builders,
• have a different name per builder,
• use the builder name in the only section of the provisioners and finally
• run packer build -only=<builder_name> to effectively limit my build to only the provisioners combined with the specific builder.

This is all fine.

What I am now trying to do, is use the same base image to create 3 different builds (and resulting AMIs). Obviously, I could just copy-paste the same builder config 3 times and then use 3 different provisioners, linking each to the respective builder, using the only parameter.

This feels totally wasteful and very error prone though... It sounds like I should be able to use the same builder and just limit which provisioners are applied .. ?

Is my only solution to use 3 copy-pasted builders? Is there any better solution?

Answer 1

only works on filters on builder name so that is not an option.

You could solve this with any of these aproches:

1. Preprocess a json and create 3 templates from one.

2. Use a template with a user variable defining which build it is and build 3 times. Use conditions on the variable in you scripts to run the correct scripts.

3. Build a base AMI with the common parts of the template and then run 3 different builds on that provisioning the differences.

In general Packer try to solve one thing well, by not including a advanced DSL for describing different build flavours the scope decreses. It's easy to preprocess and create json for more advanced use cases.

Answer 2

I had the same issue, where I want to build 2 different AMIs (one for staging, one for production) and the only difference between them is the ansible group to apply during the provisioning. Building off the answer by @Rickard ov Essen I wrote a bash script using jq to duplicate the builder section of the config.

Here's my packer.json file:

{
    "builders": [
        {
            "type": "amazon-ebs",
            "name": "staging",
            "region": "ap-southeast-2",
            "source_ami_filter": {
                "filters": {
                    "virtualization-type": "hvm",
                    "name": "ubuntu/images/hvm-ssd/ubuntu-xenial-16.04-amd64-server-*",
                    "root-device-type": "ebs"
                },
                "owners": ["099720109477"],
                "most_recent": true
            },
            "instance_type": "t2.nano",
            "ssh_username": "ubuntu",
            "force_deregister": true,
            "force_delete_snapshot": true,
            "ami_name": "my-ami-{{ build_name }}"
        }
    ],
    "provisioners": [
        {
            "type": "ansible",
            "playbook_file": "provisioning/site.yml",
            "groups": ["{{ build_name }}"]
        }
    ]
}

The ansible provisioner user the variable build_name to choose which ansible group to run.

Then I have a bash script build.sh which runs the packer build:

#!/bin/bash

jq '.builders += [.builders[0] | .name = "production"]' packer.json > packer_temp.json

packer build packer_temp.json

rm packer_temp.json

You can see what the packer_temp.json file looks like on this jqplay.

If you need to add more AMIs you can just keep adding more jq filters:

jq '.builders += [.builders[0] | .name = "production"] | .builders += [.builders[0] | .name = "test"]

This will add another AMI for test.