This is AWS EC2 AMI image builder for Arch Linux.
This builder tries to keep image as clean as possible from not needed stuff and installs just minimal set of applications needed to get bootable Arch Linux on EC2 instances.
For example, this builder doesn't even install Python. Usually Python is pulled by awslogs, aws-cli or cloud-init, but this builder uses tiny alternatives like shell-based ec2-init and aws-curl to call AWS services and to use EC2 metadata to configure ssh keys and invoke EC2 initialization commands.
The time to build AMI usually depends only on CPU, since kernel is built inside instance.
Only arm64 instances are supported at this time. If you are looking for amd64 AMI images, then they are available here.
Produced AMI can be used to spawn new instances. Produced AMIs support partially
functionality of cloud-init without pulling Python. See
ec2-init for more details.
This repo has two scripts, one for automatic bootstrap arch-ami-builder and
one for manual bootstrap arch-ami-sideload.
Below you can read about different available methods.
This is default mode and it provides a good visibility to what is happenning since bootstrap process is executed over SSH.
Script requires aws-cli and aws credentials to run (create aws resources,
including spawning EC2 instance). Grant AmazonEC2FullAccess and
IAMFullAccess to the user that will invoke the script.
Script creates all necessary resources like security group, IAM role, EC instance profile, S3 bucket for bootstrap scripts. Bootstrap scripts are uploaded over SSH to the instance and are invoked over SSH to track the progress in terminal. The process will require one reboot and script will automatically reconnect SSH session after reboot.
Once the process will be completed, the EC2 instance will be terminated.
If something will go wrong, EC2 instance could get stuck and still running. Usually, if something fails, you can see a root cause straight in the terminal. If instance doesn't boot properly, then you could use EC2 serial console access from AWS console to get know more details about what went wrong during boot.
Keep in mind, provisioning using SSH requires SSH key pair created in AWS console and installed locally so your ssh client can connect to EC2 instance.
You can run script as below to use this mode:
PROVISION_MODE=ssh EC2_KEY_PAIR="My Keys" ./arch-ami-builderThere are more options available, please see below.
This is the mode that can be used to run the bootstrap process in the cloud in detached mode.
Script requires aws-cli and aws credentials to run (create aws resources,
including spawning EC2 instance). Grant AmazonEC2FullAccess and
IAMFullAccess to the user that will invoke the script.
Script creates all necessary resources like security group, IAM role, EC instance profile, S3 bucket for bootstrap scripts. Bootstrap scripts with all configuration are uploaded to S3 bucket and EC2 instance is spawned with user data set to automatically invoke bootstrap process during cloud-init.
The process will continue on EC2 instance and you could track the progress if you need using EC2 serial console access using AWS console for initial phase.
Final phase requires using ssh access and key pair to track the progress.
Once the process will be completed, the EC2 instance will be terminated.
If something will go wrong, EC2 instance could get stuck and still running. Usually the process takes no more than 1 hour. If after 1 hour you will see that EC2 instance is still running, then use EC2 serial console to find root cause, submit an issue and terminate the instance to avoid extra charge.
You can run script as below to use this mode:
PROVISION_MODE=ec2-init ./arch-ami-builderThere are more options available, please see below.
This is minimalistic process that requires some upfront manual work, mostly to create AWS resources including spawning EC2 instance. Once EC2 is spawned the script will help to upload bootstrap scripts. The rest of the bootstrap process will be in your hands. Usually, if everything is ok, it is just as simple as running shell scripts over ssh twice (before reboot and after).
Once the process will be completed, the EC2 instance will be terminated.
-
Launch EC2 instance with following parameters:
- latest Amazon Linux 2 AMI, arm64 architecture
- any arm64 instance type, recommended is a1.2xlarge (faster) or t4g.micro (cheaper)
- security group that allows to connect from public internet on 22 port (ssh)
- 15 GB primary hard drive and 15 GB secondary hard drive (remove on shutdown)
- use key pair you have access to (to login using ssh)
- attach instance role with this policy.
-
Sideload bootstrap script:
./arch-ami-sideload ec2-user@ec2-hostname
-
You will need to trigger initial phases:
ssh -t ec2-user@ec2-hostname sudo /mnt/arch-bootstrap/bootstrap.sh
-
And, after reboot, finalize bootstrap by running remaining final phases:
ssh -t alarm@ec2-hostname sudo /mnt/arch-bootstrap/bootstrap.sh
You will likely need to edit
~/.ssh/known_hostsand remove previous key association with the host. -
Once process is completed the instance will be terminated and the image creation will be continued in background by AWS.
Configuration is performed once using environment variables during sideload.
There are two groups of environment variables available. The first group is responsible for the provisioning process and the second group is responsible for the bootstrap process itself.
Provisioning variables:
| Name | Description | Default Value | Class | Mode |
|---|---|---|---|---|
| PROVISION_MODE | Provision mode: ssh, ec2-init | ssh | main | all |
| AWS_REGION | AWS region | us-east-1 | main | all |
| EC2_ARCH | EC2 architecture: amd64, arm64 | arm64 | main | all |
| EC2_AMZN2_IMAGE_ID | EC2 Amazon Linux 2 AMI image ID | auto | debug | all |
| EC2_INSTANCE_TYPE | EC2 provisioning instance type | auto | tweak | all |
| EC2_VOLUME_SIZE | EC2 root volume size | 15GB | tweak | all |
| EC2_VOLUME_TYPE | EC2 root volume type | gp2 | tweak | all |
| EC2_KEEP_AUX_DISK | Keep aux disk volume? | false | debug | all |
| EC2_KEY_PAIR | AWS key pair name | no but required for ssh | main | all |
| EC2_SECURITY_GROUP | EC2 security group name | arch-ami-builder | tweak | all |
| EC2_ROLE | IAM role to attach to EC2 instance | arch-ami-builder | tweak | all |
| EC2_CLEANUP | Terminate instance if running | true | debug | ssh |
| SIDELOAD_S3_PATH | Sideload s3 path | arch-ami-builder-{acc}-{reg} | tweak | ec2-init |
| SIDELOAD_EC2_PATH | Sideload ec2 path | /mnt/arch-bootstrap | debug | all |
| SIDELOAD_EC2_CLEAN | Clean ec2 sideload path before? | false | debug | ssh |
Classes:
main- important parameterstweak- optional parameters needs for specific use casesdebug- just for debugging if something goes wrong
See params for bootstrap variables.
Time moves fast and AWS and Arch are too. The script that works Today could stop working Tomorrow. There is no automatic build running periodically on CI so please report any issues if you experience them.
Even if you are using ec2-init mode, it is good to have SSH access to the
instance for troubleshooting, so don't forget to pass EC2_KEY_PAIR.
EC2 serial console is also a good troubleshooting friend, especially for
ec2-init mode.
In any case:
- Try to find root cause (terminal logs, ec2 serial logs etc).
- Fix it if you can and submit PR ;-). PRs are very welcome!
- If you can't fix then submit issue and maintainers will take a look.
There are a bunch of improvements could be done. Take a look on TODO.md.
PRs are very welcome!
MIT License