BigData Helm charts

Curated Big Data charts for Kubernetes.

Install chart from helm repository

charts in `charts/`` folder are packaged and available at Gradiant's helm repo:

https://gradiant.github.io/bigdata-charts/

You can add the helm repo to your Helm CLI:

helm repo add bigdata-gradiant https://gradiant.github.io/bigdata-charts/

Then you have a collection of charts available to install. For example, to install hdfs:

helm install --name hdfs bigdata-gradiant/hdfs

Install chart from release

install using the URL of the release. For example, to install hdfs v0.1.0 chart:

helm install --name hdfs https://github.com/Gradiant/bigdata-charts/releases/download/hdfs-0.1.0/hdfs-0.1.0.tgz

Development

clone repo
adjust given chart
bump chart version if required
run tests
create pull request with issue id, attach test results if possible

Requirements

linting requires docker
running install or lint-and-install requires access to the kubernetes cluster - can be minikube, kubernetes-in-docker, or real cluster
tee console tool to output to the console and file in the same time

Linting and testing full deployment

Test specific chart, lint and install, send console logs also to the log file reports/hdfs.log:

scripts/ct.sh lint-and-install \
    --charts charts/hdfs/ \
    --chart-repos incubator=https://kubernetes-charts-incubator.storage.googleapis.com/,gradiant=https://gradiant.github.io/bigdata-charts \
    | tee reports/hdfs.log

More advanced

Output to a reports/hdfs.log,
Create charts/chart-name/ci/gke-values.yaml with your custom values, example below:

---
persistence:
  nameNode:
    enabled: true
    storageClass: standard-ssd
    accessMode: ReadWriteOnce
    size: 50Gi
  dataNode:
    enabled: true
    storageClass: standard-hdd
    accessMode: ReadWriteOnce
    size: 200Gi

You can provide multiple files in ci/ folder - for each ci/*-values.yaml there will be separate deployment
Execute full test with linting, installing, upgrading:

scripts/ct.sh lint-and-install \
    --check-version-increment \
    --upgrade \
    --charts charts/hdfs/ \
    --chart-repos incubator=https://kubernetes-charts-incubator.storage.googleapis.com/,gradiant=https://gradiant.github.io/bigdata-charts \
    | tee reports/hdfs.log

See reports/ file for generated console output.
Ensure to remove any secret data when uploading logs.

Gradiant / bigdata-charts