Table of Contents generated with DocToc
Kubernetes Cluster API Provider IBM Cloud
This repository hosts a concrete implementation of an IBM Cloud provider for the cluster-api project.
What is the Cluster API Provider IBM Cloud
The Cluster API brings declarative, Kubernetes-style APIs to cluster creation, configuration and management. The API itself is shared across multiple cloud providers allowing for true IBM Cloud hybrid deployments of Kubernetes.
Community, discussion, contribution, and support
Learn how to engage with the Kubernetes community on the community page.
You can reach the maintainers of this project at:
Code of conduct
Participation in the Kubernetes community is governed by the Kubernetes Code of Conduct.
Getting Started
Prerequisites
-
Install
kubectl
(see here). Becausekustomize
was included intokubectl
and it's used bycluster-api-provider-ibmcloud
in generating yaml files, so version1.14.0+
ofkubectl
is required, see integrate kustomize into kubectl for more info. -
You can use either VM, container or existing Kubernetes cluster act as the bootstrap cluster.
-
Install a driver if you are using minikube. For Linux, we recommend kvm2. For MacOS, we recommend VirtualBox.
-
An appropriately configured Go development environment
-
Build the
clusterctl
toolgit clone https://github.com/kubernetes-sigs/cluster-api-provider-ibmcloud $GOPATH/src/sigs.k8s.io/cluster-api-provider-ibmcloud cd $GOPATH/src/sigs.k8s.io/cluster-api-provider-ibmcloud/cmd/clusterctl go build
Cluster Creation
-
Create the
cluster.yaml
,machines.yaml
,provider-components.yaml
, andaddons.yaml
files if needed.cd examples/ibmcloud ./generate-yaml.sh [options] <path/to/clouds.yaml> <provider os: [centos,ubuntu,coreos]> cd ../..
<clouds.yaml>
is a yaml file to record how to interact with IBM Cloud, there's a sample clouds.yaml.<provider os>
specifies the operating system of the virtual machines Kubernetes will run on. Supported Operating Systems:ubuntu
centos
(Not Implemented)coreos
(Not Implemented)
Example command:
./generate-yaml.sh ./clouds.yaml ubuntu
Quick notes on clouds.yaml
$ cat clouds.yaml clouds: ibmcloud: auth: apiUserName: "Your API Username" authenticationKey: "Your API Authentication Key"
You can get
apiUserName
andauthenticationKey
from https://control.softlayer.com/ .- Logon to https://control.softlayer.com/ .
- Click your user name on the right top of the console.
- The console will navigate you to the page of
Edit User Profile
. - Scroll down the page to the bottom, you will see a section
API Access Information
. You can getapiUserName
andauthenticationKey
from there.
Special notes on SSH keys and fetchingadmin.conf
In order to allow
clusterctl
to fetch Kubernetes'admin.conf
from the master node, you must manually create the SSH key in IBM Cloud. By default the generatedmachine.yaml
usescluster-api-provider-ibmcloud
to be thesshKeyName
. However, you are free to change that.For the SSH key, you can logon to https://control.softlayer.com/ , click
Devices->Manage->SSH Keys
, you will be navigated to the page ofSSH Keys
, clickAdd
to create your own key.NOTE The private key used to connect to the master node is by default at
~/.ssh/id_ibmcloud
. It is generated bygenerate-yaml.sh
if related keys do not exist in your environment. Please make sure the content of the corresponding public key file~/.ssh/id_ibmcloud.pub
is used in creating your IBM Cloud SSH key. Environment variableIBMCLOUD_HOST_SSH_PRIVATE_FILE
is supported to customize SSH key file, set this value before you rungenerate-yaml.sh
andclusterctl
to modify default SSH key file.For the
sshUserName
, you can useroot
orubuntu
user for theUbuntu
cloud image, while you can only useroot
user for other Linux cloud images. By default the generatedmachine.yaml
usesroot
to be thesshUserName
.Optionally, add a
addons.yaml
can provide additional add ons in target cluster, for example, download k8s dashboard toexamples/ibmcloud/out/
directory:wget -O addons.yaml https://raw.githubusercontent.com/kubernetes/dashboard/master/aio/deploy/recommended/kubernetes-dashboard.yaml
Add
-a examples/ibmcloud/out/addons.yaml
inclusterctl
command, after cluster created. The dashboard pod will be created and user is able to logon through k8s dashboard:# kubectl --kubeconfig=kubeconfig get pods --all-namespaces NAMESPACE NAME READY STATUS RESTARTS AGE ... kube-system kubernetes-dashboard-5f7b999d65-ntrxb 1/1 Running 0 12m ...
-
Create a cluster:
You should review files cluster.yaml
and machines.yaml
, and ensure the domain
, dataCenter
and osReferenceCode
are set. You can customize those values based on your requirement.
For machines.yaml
, you can follow the sample file, and make sure sshKeyName
and the Label
value in SSH Keys
are identical, shown as below:
- If you are using minikube:
./clusterctl create cluster --bootstrap-type minikube --bootstrap-flags kubernetes-version=v1.12.3 \
--provider ibmcloud -c examples/ibmcloud/out/cluster.yaml \
-m examples/ibmcloud/out/machines.yaml -p examples/ibmcloud/out/provider-components.yaml \
-a examples/ibmcloud/out/addons.yaml
To choose a specific minikube driver, please use the --bootstrap-flags vm-driver=xxx
command line parameter. For example to use the kvm2 driver with clusterctl you woud add --bootstrap-flags vm-driver=kvm2
, for linux, if you haven't installed any driver, you can add --bootstrap-flags vm-driver=none
.
- If you are using kind:
./clusterctl create cluster --bootstrap-type kind --provider ibmcloud \
-c examples/ibmcloud/out/cluster.yaml -m examples/ibmcloud/out/machines.yaml \
-p examples/ibmcloud/out/provider-components.yaml -a examples/ibmcloud/out/addons.yaml
NOTE: If you have an existing Kubernetes cluster which is created by kind, and the cluster name is clusterapi
, you should delete this cluster before run clusterctl
to create a cluster, for example:
# kind get clusters
clusterapi
# kind delete cluster --name=clusterapi
Deleting cluster "clusterapi" ...
- If you are using existing Kubernetes cluster:
./clusterctl create cluster --bootstrap-cluster-kubeconfig ~/.kube/config \
--provider ibmcloud -c examples/ibmcloud/out/cluster.yaml \
-m examples/ibmcloud/out/machines.yaml -p examples/ibmcloud/out/provider-components.yaml \
-a examples/ibmcloud/out/addons.yaml
For the above command, the bootstrap-cluster-kubeconfig
was located at ~/.kube/config
, you must update it
to use your kubeconfig.
Additional advanced flags can be found via help.
./clusterctl create cluster --help
Interacting with your cluster
If you are using kind, config the KUBECONFIG
first before using kubectl:
export KUBECONFIG="$(kind get kubeconfig-path --name="clusterapi")"
Once you have created a cluster, you can interact with the cluster and machine resources using kubectl:
# kubectl --kubeconfig=kubeconfig get clusters
NAME AGE
test1 28m
# kubectl --kubeconfig=kubeconfig get machines
NAME PROVIDERID PHASE
ibmcloud-master-464lh ibmcloud:////82692207 Running
ibmcloud-node-rjtnv ibmcloud:////82692501 Running
# kubectl --kubeconfig=kubeconfig get machines -o yaml
apiVersion: v1
items:
- apiVersion: cluster.k8s.io/v1alpha1
kind: Machine
metadata:
annotations:
ibmcloud-ip-address: 158.85.27.183
creationTimestamp: "2019-06-15T15:56:16Z"
finalizers:
- foregroundDeletion
- machine.cluster.k8s.io
generateName: ibmcloud-master-
generation: 1
labels:
cluster.k8s.io/cluster-name: test1
...
Cluster Deletion
This guide explains how to delete all resources that were created as part of your IBM Cloud Cluster API Kubernetes cluster.
-
Delete all of the node Machines in the cluster. Make sure to wait for the corresponding Nodes to be deleted before moving onto the next step. After this step, the master node will be the only remaining node.
kubectl --kubeconfig=kubeconfig delete machines -l set=node kubectl --kubeconfig=kubeconfig get nodes
-
Delete the master machine.
kubectl --kubeconfig=kubeconfig delete machines -l set=master
-
Delete the kubeconfig file that were created for your cluster.
rm kubeconfig
-
Delete the SSH keypair that were created for your cluster machine.
rm -rf $HOME/.ssh/id_ibmcloud*
Create additional worker nodes
Please refer to Create a new worker node for further info.
How to use clusterctl image
The quay.io/cluster-api-provider-ibmcloud/clusterctl
image is designed to run independently to provision ibmcloud cluster. We have embedded the kind
and kubectl
into clusterctl
image. Please refer to How to use clusterctl image for further info.
Trouble shooting
Please refer to Trouble shooting documentation for further info.
IBM Cloud References
If you have any further question about IBM Cloud settings in provider, please refer to IBM Cloud help documentation for further information.