Multiple EC2 instances built when running from Bitbucket Pipelines

[Steven Roomberg]

Overview of the Issue

When I run packer builds from Bitbucket Pipelines, 2 instances are consistently spun up even though packer only runs on one of them. Since the packer process doesn't seem to recognize the unused instance, this resource (and the associated packer resources) are left behind and not cleaned up.

This does not occur when running from a local machine so my guess is there is a latency or promise/future issue somewhere and a retry in the packer code is causing a second instance to be spun up before the a response can be received from the first instance (which eventually gets used).

Plugin and Packer version

amazon-ebs, 1.1.3 (also received same result with latest version 1.2.6)

Packer Logs

Here is the command line output from the packer build command from Bitbucket Pipelines:

chaosfarmami.amazon-ebs.ubuntu-us-east-1: output will be in this color.
==> ami.amazon-ebs.ubuntu-us-east-1: Prevalidating any provided VPC information
==> ami.amazon-ebs.ubuntu-us-east-1: Prevalidating AMI Name: ami-10072023-121427
        ami.amazon-ebs.ubuntu-us-east-1: Found Image ID: ami-089b5711e63812c2a
==> ami.amazon-ebs.ubuntu-us-east-1: Creating temporary keypair: packer_64ac2e66-13e8-9f7c-fd2b-a19e53970c18
==> ami.amazon-ebs.ubuntu-us-east-1: Creating temporary security group for this instance: packer_64ac2e67-e729-055b-f474-48f9cb227df0
==> ami.amazon-ebs.ubuntu-us-east-1: Authorizing access to port 22 from [0.0.0.0/0] in the temporary security groups...
==> ami.amazon-ebs.ubuntu-us-east-1: Launching a source AWS instance...
==> ami.amazon-ebs.ubuntu-us-east-1: Adding tags to source instance
        ami.amazon-ebs.ubuntu-us-east-1: Adding tag: "Name": "ami-10072023-121427"
        ami.amazon-ebs.ubuntu-us-east-1: Instance ID: i-0d1d1e28209854e1e

AWS Console Screenshot

Simplified Packer Buildfile

{
    "packer": [
        {
            "required_plugins": [
                {
                    "amazon-ebs": {
                        "version": "= 1.1.3",
                        "source": "github.com/hashicorp/amazon"
                    }
                }
            ]
        }
    ],
    "source": [
        {
            "amazon-ebs": {
                "ubuntu-us-east-1": {
                    "ami_name": "ami-10072023-114136",
                    "region": "us-east-1",
                    "access_key": "...",
                    "secret_key": "...",
                    "token": "...",
                    "launch_block_device_mappings": [
                        {
                            "delete_on_termination": true,
                            "device_name": "/dev/sda1",
                            "iops": 3000,
                            "throughput": 125,
                            "volume_type": "gp3",
                            "volume_size": 8
                        }
                    ],
                    "tags": {
                        "Name": "ami-10072023-114136",
                    },
                    "source_ami": "ami-089b5711e63812c2a",
                    "instance_type": "t2.xlarge",
                    "ssh_username": "ubuntu",
                    "vpc_id": "...",
                    "subnet_id": "...",
                    "ami_users": [...],
                    "ami_regions": [...],
                    "skip_region_validation": true,
                    "snapshot_users": [...],
                    "run_tags": {
                        "Name": "ami-10072023-114136",
                    },
                    "shutdown_behavior": "terminate",
                    "max_retries": 10
                }
            }
        }
    ],
    "build": [
        {
            "name": "ami",
            "sources": [
                "source.amazon-ebs.ubuntu-us-east-1"
            ],
            "provisioner": [...],
            "post-processors": [...]
        }
    ]
}