Install KubeSphere on Existing Kubernetes Cluster

English | 中文

In addition to supporting deploying on VM and BM, KubeSphere also supports installing on cloud-hosted and on-premises existing Kubernetes clusters.

Prerequisites

Kubernetes Version: 1.15.x, 1.16.x, 1.17.x;

Helm Version: >= 2.10.0 (excluding 2.16.0 and 2.16.5) and < 3.0, see Install and Configure Helm in Kubernetes;

CPU > 1 Core, Memory > 2 G;

An existing default Storage Class in your Kubernetes clusters.

The CSR signing feature is activated in kube-apiserver when it is started with the --cluster-signing-cert-file and --cluster-signing-key-file parameters, see RKE installation issue.

Make sure your Kubernetes version is compatible by running kubectl version in your cluster node. The output looks as the following:

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.1", GitCommit:"4485c6f18cee9a5d3c3b4e523bd27972b1b53892", GitTreeState:"clean", BuildDate:"2019-07-18T09:09:21Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.1", GitCommit:"4485c6f18cee9a5d3c3b4e523bd27972b1b53892", GitTreeState:"clean", BuildDate:"2019-07-18T09:09:21Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"}

Note: Pay attention to Server Version line, if GitVersion is greater than v1.15.0, it's good to go. Otherwise you need to upgrade your kubernetes first.

Make sure you've already installed Helm, and it's version is greater than 2.10.0. You can run helm version to check. The output looks like below:

$ helm version
Client: &version.Version{SemVer:"v2.13.1", GitCommit:"618447cbf203d147601b4b9bd7f8c37a5d39fbb4", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.13.1", GitCommit:"618447cbf203d147601b4b9bd7f8c37a5d39fbb4", GitTreeState:"clean"}

Note: If you get helm: command not found, it means Helm is not installed yet. You can refer to Install Helm to find out how to install Helm, and don't forget to run helm init first after installation. If you use an older version (<2.10.0), you need to Upgrade Helm and Tiller.

Check if the available resources meet the minimal prerequisite in your cluster.

$ free -g
              total        used        free      shared  buff/cache   available
Mem:              16          4          10           0           3           2
Swap:             0           0           0

Check if there is a default Storage Class in your cluster. An existing Storage Class is the prerequisite for KubeSphere installation.

$ kubectl get sc
NAME                      PROVISIONER               AGE
glusterfs (default)               kubernetes.io/glusterfs   3d4h

If your Kubernetes cluster environment meets all requirements mentioned above, then you can start to install KubeSphere.

To Start Deploying KubeSphere

Minimal Installation

Attention: Following section is only used for minimal installation by default. KubeSphere has decoupled some core components since v2.1.0, for more pluggable components installation, see Enable Pluggable Components and Configuration Table.

kubectl apply -f https://raw.githubusercontent.com/kubesphere/ks-installer/master/kubesphere-minimal.yaml

Then inspect the logs of installation.

kubectl logs -n kubesphere-system $(kubectl get pod -n kubesphere-system -l app=ks-install -o jsonpath='{.items[0].metadata.name}') -f

When all Pods of KubeSphere are running, it means the installation is successful. Check the port (30880 by default) of the console service by the following command. Then you can use http://IP:30880 to access the console with the default account admin/P@88w0rd.

kubectl get svc/ks-console -n kubesphere-system

Enable Pluggable Components

Attention: make sure there is enough CPU and memory available in your cluster. See the Configuration Table.

[Optional] Create the secret of certificate for Etcd in your Kubernetes cluster. This step is only needed when you want to enable Etcd monitoring.

Note: Create the secret according to the actual Etcd certificate path of your cluster; If the Etcd has not been configured certificate, an empty secret needs to be created.

If the Etcd has been configured with certificates, refer to the following step (The following command is an example that is only used for the cluster created by kubeadm):

$ kubectl -n kubesphere-monitoring-system create secret generic kube-etcd-client-certs  \
--from-file=etcd-client-ca.crt=/etc/kubernetes/pki/etcd/ca.crt  \
--from-file=etcd-client.crt=/etc/kubernetes/pki/etcd/healthcheck-client.crt  \
--from-file=etcd-client.key=/etc/kubernetes/pki/etcd/healthcheck-client.key

If the Etcd has not been configured with certificates.

kubectl -n kubesphere-monitoring-system create secret generic kube-etcd-client-certs

If you already have a minimal KubeSphere setup, you still can enable the pluggable components by editing the ConfigMap of ks-installer using the following command.

Note: Please make sure there is enough CPU and memory available in your cluster, see the Configuration Table for reference.

kubectl edit cm ks-installer -n kubesphere-system

Inspect the logs of installation.

kubectl logs -n kubesphere-system $(kubectl get pod -n kubesphere-system -l app=ks-install -o jsonpath='{.items[0].metadata.name}') -f

Upgrade

Download the Yaml file as follows:

wget https://raw.githubusercontent.com/kubesphere/ks-installer/master/kubesphere-minimal.yaml

Sync the changes from the old version to 2.1.1 in the config section of kubesphere-minimal.yaml, note the storage class and the pluggable components need to be consistent with the old version:

kubectl apply -f kubesphere-minimal.yaml

Note: GitLab and Harbor are not included in 2.1.1, please refer to Harbor Documentation and Gitlab Documentation to install them if needed.

Configuration Table

Pay attention to the resource request in the first column. You need to make sure there is enough CPU and memory available in your cluster, especially for enabling Logging, DevOps, Istio, Harbor and GitLab.

Parameter		Description	Default
persistence	storageClass	Installer will use the default StorageClass, you can also designate another StorageClass	“”
etcd	monitoring	Whether to enable etcd monitoring	False
	endpointIps	etcd address（for etcd cluster, see an example value like `192.168.0.7,192.168.0.8,192.168.0.9`）
	port	etcd port (Default port: 2379, you can appoint any other port)	2379
	tlsEnable	Whether to enable etcd TLS certificate authentication.（True / False）	True
common	mysqlVolumeSize	MySQL volume size (cannot be modified after set)	20Gi
	minioVolumeSize	Minio volume size (cannot be modified after set)	20Gi
	etcdVolumeSize	etcd volume size (cannot be modified after set)	20Gi
	openldapVolumeSize	openldap volume size (cannot be modified after set)	2Gi
	redisVolumSize	redis volume size (cannot be modified after set)	2Gi
console	enableMultiLogin	Whether to enable multiple point login of one account（True / False）	False
console	port	Console Port（NodePort）	30880
monitoring	prometheusReplicas	Prometheus replicas	1
	prometheusMemoryRequest	Prometheus memory request	400Mi
	prometheusVolumeSize	Prometheus volume size	20Gi
	grafana.enabled	Whether to enable Grafana installation（True / False）	False
openpitrix (at least 0.3 core, 300 MiB)	enable	App store and app templates are based on OpenPitrix, it's recommended to enable OpenPitrix installation（True / False）	False
logging (at least 56 M, 2.76 G)	enabled	Whether to enable logging system installation （True / False）	False
	elasticsearchMasterReplicas	Elasticsearch master replicas	1
	elasticsearchDataReplicas	Elasticsearch data replicas	1
	logsidecarReplicas	Logsidecar replicas	2
	elasticsearchVolumeSize	ElasticSearch volume size	20Gi
	logMaxAge	How many days the logs are remained	7
	elkPrefix	Log index	logstash
	containersLogMountedPath	The mounting path of container logs	“”
	kibana.enabled	Whether to enable Kibana installation （True / False）	False
devops (at least 0.47 core, 8.6 G for multi-node cluster)	enabled	Whether to enable DevOps system installation （True / False）	False
	jenkinsMemoryLim	Jenkins Memory Limit	2Gi
	jenkinsMemoryReq	Jenkins Memory Request	1500Mi
	jenkinsVolumeSize	Jenkins volume size	8Gi
	jenkinsJavaOpts_Xms	Jenkins JVM parameter （Xms）	512m
	jenkinsJavaOpts_Xmx	Jenkins JVM parameter（Xmx）	512m
	jenkinsJavaOpts_MaxRAM	Jenkins JVM parameter（MaxRAM）	2Gi
	sonarqube.enabled	Whether to install SonarQube（True / False）	False
metrics_server (at least 5 m, 44.35 MiB)	enabled	Whether to install metrics_server （True / False）	False
servicemesh (at least 2 core, 3.6 G)	enabled	Whether to install Istio （True / False）	False
notification (Notification and Alerting together, at least 0.08 core, 80 M)	enabled	Whether to install Notification sysytem （True / False）	False
alerting	enabled	Whether to install Alerting sysytem （True / False）	False

Support, Discussion, and Community

If you need any help with KubeSphere, please join us at Slack Channel.

cypctlinux / ks-installer