This repository consists of an AWS CloudFormation template that may be used to deploy a GitLab runner with Docker executor and auto-scaling based on number of builds on AWS.
The runners have a shared cache to speed up builds. Objects in the bucket are automatically expired after a configurable number of days — 0 means that cache objects will never expire.
- 1 S3 bucket to store runners' cache.
- 1 EC2 instance that is the runners' manager: it invokes AWS APIs to spawn
and terminate other EC2 instances (via
docker-machine) and runs Docker containers on them to process GitLab CI builds.
If you want to create the stack within a specific VPC, you'll first need to create it manualy.
Please note that docker-machine uses availability zone A per default therefore you'll need to have at minimum the 'A' zone available in your vpc.
- Note: This is something you can contribute to and propose a change request to add an availability zone selector...
When you launch the stack you are required to pass a GitLab Runner token. This is not to be confused with a GitLab Runner registration token!
You can obtain a registration token by navigating to the "Settings › CI / CD" page of any project for which you have administrative rights. It'll be available under "Runners settings".
You can then obtain a GitLab Runner token by using the (undocumented) endpoint
POST /runners:
# Assuming the GitLab instance is available at https://gitlab.example.org
# and the GitLab Runner registration token is "abcdef1234567890":
curl -XPOST -H 'Content-Type: application/json' -H 'Accept: application/json' \
-d '{"token":"abcdef1234567890","run_untagged":true,"locked":false}' \
https://gitlab.example.org/api/v4/runnersIf everything goes fine, the response will be a JSON that has a token key:
this is the GitLab Runner token you were looking for.
Credentials must be rotated, and humans must remember to rotate credentials. But credentials are not always strictly necessary.
The runners' manager instance has an AWS Instance Profile attached that makes it possible to invoke EC2 and S3 APIs using dynamically obtained credentials, that have a short lifetime and therefore don't need to be rotated. Thus, no IAM access keys are involved in this stack — except the ones you may use to create or update the stack using the AWS APIs or the CLI, of course.
The only credential that is actually stored somewhere is the GitLab CI token. If stolen, it would allow a malicious user to "intercept" your builds and run them on their infrastructure, exposing other secrets as a consequence. You should treat this as a very sensitive information.
This stack doesn't provide any special security measure: the value is passed as
plain text to CloudFormation at stack creation as a "sensitive parameter"
(NoEcho: true), and is stored in plain text on the runners' manager in a file
that is readable only by gitlab-runner user. The provisioning of said file
happens via cfn-init. The value is then used by GitLab Runner itself,
presumably in HTTP-over-TLS communications with the GitLab instance.
At stack creation you are required to specify an AWS Key Pair to provide access to the runners' manager instance. When the stack is created, you can access your instance with the following command:
ssh -i /PATH/TO/IDENTITY_FILE ec2-user@INSTANCE-PUBLIC-IPThe runners' manager Security Group allows connections on port 22 by any IPv4
address (CIDR: 0.0.0.0/0), and all other ports are unaccessible. This is far
from being an optimal solution but, since SSH authentication requires an SSH key
pair, it should be pretty safe anyways. Counter-measures like Fail2Ban are
not deployed out of the box, either.