Cross Cloud Continuous Integration
This cross-cloud project aims to demonstrate cross-project compatibility in the CNCF by building, E2E testing and deploying selected CNCF projects to multiple clouds using continuous integration (CI). The initial proof of concept is being done by deploying a Kubernetes cluster with CoreDNS and Prometheus to AWS and Packet. The eventual goal is to support all CNCF projects on AWS, Packet, GCE, GKE, Bluemix and Azure.
Our cross-cloud provisioning is accomplished with the terraform modules for AWS, Azure, GCE, GKE, Packet which deploy kubernetes using a common set of variables producing KUBECONFIGs for each.
We fork CNCF projects into the CNCF.ci gitlab server (https://gitlab.cncf.ci/) where we add a .gitlab-ci.yml config file to drive the CI process.
Each push to projects branches we are interested in will generate binaries, tests, and container images exported as ci variables that will allow continous deployment of that projects artifacts.
Feedback on our approach to building and e2e testing welcome:
- ii/kubernetes#1 => https://gitlab.cncf.ci/kubernetes/kubernetes/blob/ci-master/.gitlab-ci.yml
- ii/coredns#1 => https://gitlab.cncf.ci/coredns/coredns/blob/ci-master/.gitlab-ci.yml
- ii/prometheus#1 https://gitlab.cncf.ci/prometheus/prometheus/blob/ci-master/.gitlab-ci.yml
- ii/alertmanager#1 => https://gitlab.cncf.ci/prometheus/alertmanager/blob/ci-master/.gitlab-ci.yml
- ii/node_exporter#1 => https://gitlab.cncf.ci/prometheus/node_exporter/blob/ci-master/.gitlab-ci.yml
In our first stage we tie together a particular set of project branches (stable,master,alpha) and collect the release.env
We override these variables for the sets of branches we want to test together:
variables:
K8S_BRANCH: ci-master
COREDNS_BRANCH: ci-master
PROMETHEUS_BRANCH: ci-master
NODE_EXPORTER_BRANCH: ci-master
ALERT_MANAGER_BRANCH: ci-master
ALERT_MANAGER_IMAGE=registry.cncf.ci/prometheus/alertmanager
ALERT_MANAGER_TAG=ci-v0.6.2.e9aefe84.5699
COREDNS_IMAGE=registry.cncf.ci/coredns/coredns
COREDNS_TAG=ci-v007.97c730ab.5713
KUBERNETES_IMAGE=registry.cncf.ci/kubernetes/kubernetes/hyperkube-amd64
KUBERNETES_TAG=ci-v1-6-3.job.5949
NODE_EXPORTER_IMAGE=registry.cncf.ci/prometheus/node_exporter
NODE_EXPORTER_TAG=ci-v0.14.0.ae1e00a5.5696
PROMETHEUS_IMAGE=registry.cncf.ci/prometheus/prometheus
PROMETHEUS_TAG=ci-v1.6.3.fd07e315.6097
The Cross cloud repo itself includes a container which is used to deploy k8s on several clouds, making the KUBECONFIG available for cross-project deployments.
We are currently using helm and overriding the image_url and image_tag with our CI generated artifacts. and then runs runs E2E tests on the whole infrastructure
Initially we are testing head and the last stable release of Kubernetes, CoreDNS and Prometheus.
