Enviroment setup

Bootstrapper will be running on a machine(AKA: bootstrapper server), which need to meet the following requirements

The kubernetes machines waiting for install need to be connected with bootstrapper server.
Bootstrapper server is a linux server with docker daemon(1.11 or later) installed.
Have root access of the bootstrapper server.

Configurations and download image files that bootstrapper needs.

The following steps will prepare the environment, generate configurations and build docker images.

if there's no internet access on the bootstrapper server, you can copy the pre-donwloaded /bsroot directory to it.

After getting the sextant code, you need to plan the cluster installation details by editing cloud-config-server/template/cluster-desc.sample.yaml. Then build bootstrapper to the ./bsroot directory.

go get -u -d github.com/k8sp/sextant/...
cd $GOPATH/src/github.com/k8sp/sextant
vim golang/template/cluster-desc.sample.yaml
./bsroot.sh golang/template/cluster-desc.sample.yaml

Uploaded to the bootstrapper server

If the above steps is done on the bootstrapper server, you can skip this step.

Packing direcotry ./bsroot: tar czvf bsroot.tar.gz ./bsroot
Upload bsroot.tar.gz to the bootstrapper server.(using tools such as SCP or FTP)
Extract bsroot.tar.gz to / directory on bootstrapper server.

Start bootstrapper

ssh root@bootstrapper
cd /bsroot
./start_bootstrapper_container.sh /bsroot

Setup kubernetes cluster using the bootstrapper

Just set kubernetes nodes boot through PXE, reboot the machine, then it will completed Kubernetes and Ceph installation automatically.

Using kubernetes cluster

Configurate kubectl client

scp root@bootstrapper:/bsroot/setup-kubectl.bash ./
./setup-kubectl.bash

Verify kubectl configuration and connection

Execute the following command, verify whether the client has been property configured according to the return result.

bootstrapper ~ # ./kubectl get nodes
NAME                STATUS                     AGE
08-00-27-4a-2d-a1   Ready,SchedulingDisabled   1m

Using Ceph cluster

After the cluster installation is complete, you can use the following command to obtain admin keyring for the later use.

etcdctl --endpoints http://08-00-27-ef-d2-12:2379 get /ceph-config/ceph/adminKeyring

For example, mount a directory with CephFS.

mount -t ceph 192.168.8.112:/ /ceph -o name=admin,secret=[your secret]

Cluster maintenance

How to updating the cert after the cluster is running for some time.

Edit the confuration openssl.cnf in certgen.go.

[req]
req_extensions = v3_req
distinguished_name = req_distinguished_name
[req_distinguished_name]
[ v3_req ]
basicConstraints = CA:FALSE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
subjectAltName = @alt_names
[alt_names]
DNS.1 = kubernetes
DNS.2 = kubernetes.default
DNS.3 = kubernetes.default.svc
DNS.4 = kubernetes.default.svc.cluster.local
DNS.5 = 10.10.10.201
IP.1 = 10.100.0.1

Regenerating api-server.pem and other files according the openssl.cnf: https://coreos.com/kubernetes/docs/latest/openssl.html
Restart master processes, including api-server,controller-manager,scheduler,kube-proxy
Delete default secret under kube-system/default namesapce using kubectl delete secret
Resubmit failed service.

jljlpch / sextant