helm needs socat on the nodes of the k8s cluster
sebgoa opened this issue · comments
But cluster nodes, may not have socat installed by default.
For instance coreOS stable does not ship socat:
This is a problem since helm use a port-forwarding technique to connect to tiller.
ps: there might be the same issue with nsenter
That's not so much a Helm problem, as a Kubernetes problem (https://github.com/kubernetes/kubernetes/blob/master/pkg/kubelet/dockertools/docker_manager.go#L1167)
Personally I ended up bootstrapping all of our CoreOs nodes with socat in /opt/bin and adding /opt/bin to kubelet service PATH environment variable with a systemd drop-in.
Yeah, I think we are going to have to bump this up to the CoreOS/Kubernetes level. However, I'm leaving the issue open here while we track what's going on upstream.
Yes, I did not mean to say it was a Helm problem. But we as Helm are assuming that k8s cluster nodes will have socat...which may not be true.
Honestly I think protobuf is cool, but for exposing API outside k8s it makes things a bit tricky.
@maratoid where did you get the socat binary? I tried the statically built from https://github.com/andrew-d/static-binaries but it almost crashes VM with a very high CPU usage when I do helm install some chart
@maratoid this is what I was testing with helm v2, socat from there works fine only with kubectl port-forward but with helm socat goes nuts of using a lot of CPU and helm install just hangs
FWIW, in Slack, dhilipkumars hit this issue when running Kubernetes on Ubuntu locally. The fix was to export HELM_HOST to the k8s API address/port. We need a FAQ or something to document issues like this.
CoreOS doesn't ship socat but the container that the kubelet runs in has ./usr/bin/socat. What is the issue with that socat? How do I reproduce?
cc @pbx0 @aaronlevy @derekparker
@philips if kubernetes e.g. is run via go binaries then kubectl port-forward and helm v2 install chart do not work
@philips the issue I faced was the following. I have a coreOS k8s cluster with coreOS 899.
./helm init deploy the tiller server fine, but by default the helm client tries to setup a port-forward to the tiller pod. port-forward uses socat, hence helm fails talking to tiller because socat is not found. I personally setup a NodePort service and set my HELM_HOST to that port, but it is insecure.
@rimusz OK, CoreOS Linux won't support this setup because kubelet has a number of dependencies that aren't. Please run kubelet in a container.
@Runseb I am able to successfully use helm v2.0.0-alpha.4 with CoreOS Linux and Kubernetes v1.4-beta0:
$ helm install docs/examples/alpine
Error: file "docs/examples/alpine" not found
$ cd
$ cd src/k8s.io/he
heapster/ helm/
$ cd src/k8s.io/helm/
$ helm install docs/examples/alpine
washing-squid
Last Deployed: Fri Sep 9 00:08:03 2016
Namespace: default
Status: DEPLOYED
Resources:
==> v1/Pod
NAME READY STATUS RESTARTS AGE
washing-squid-my-alpine 0/1 ContainerCreating 0 1s
@Runseb alpha using the hyperkube kubelet container
@Runseb CoreOS 1153.0.0 to be precise
thanks, I will try that one
sorry for the delay. Indeed running the kubelet with the wrapper solves this.
Check kubelet wrapper for example of unit file to run the wrapper.
tested under coreOS stable 1122 and helm from HEAD.
thx @philips
I am closing this.
The documentation for the kubelet makes it seem like running the kubelet in a docker container is not a good idea for production.
--containerized Experimental support for running kubelet in a container. Intended for testing. [default=false]
http://kubernetes.io/docs/admin/kubelet/
Is the kubelet production ready?
If you don't want to run kubelet in a container you may install socat in /opt/bin/ and add it to the PATH of the kubelet systemd.unit
Environment="PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/bin"
I use @mattthias approach too
I was testing helm on ubuntu VM and faced same issue and resolved after installing socat and nsenter
For helm to properly work, you need socat and nsenter installed on all nodes
for socat install use:
sudo apt-get update && sudo apt-get install socat
for nsenter install use :
cd /tmp; curl https://www.kernel.org/pub/linux/utils/util-linux/v2.25/util-linux-2.25.tar.gz | tar -zxf-; cd util-linux-2.25;
sudo apt-get install autopoint autoconf libtool automake
./configure --without-python --disable-all-programs --enable-nsenter --without-ncurses
make nsenter; cp nsenter /usr/local/bin