stefanprodan / k8s-prom-hpa

Kubernetes Horizontal Pod Autoscaler with Prometheus custom metrics

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

HPA not showing memory, and showing wrong CPU

lukebond opened this issue · comments

Steps to reproduce:

$ kubectl create -f metrics-server/
$ kubectl create -f ./podinfo/podinfo-svc.yaml,./podinfo/podinfo-dep.yaml
$ kubectl create -f ./podinfo/podinfo-hpa.yaml

Soon after, I start seeing erroneous values for the CPU (they're not 100%, I've not yet started sending any traffic); and the memory is showing as <unknown>.

$ kubectl get hpa -w
NAME      REFERENCE            TARGETS                     MINPODS   MAXPODS   REPLICAS   AGE
podinfo   Deployment/podinfo   <unknown>/200Mi, 100%/80%   2         10        4          2m10s

If I look at the logs of the metrics server I see the following:

$ kubectl logs metrics-server-68d85f76bb-kgqlr -n kube-system
E0320 14:29:47.201497       1 reststorage.go:144] unable to fetch pod metrics for pod default/podinfo-7b8c9bc5c9-d5f42: no metrics known for pod

I'm running on CentoOS on clusters created with kubeadm init with just the defaults.

$ kubeadm version
kubeadm version: &version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.4", GitCommit:"c27b913fddd1a6c480c229191a087698aa92f0b1", GitTreeState:"clean", BuildDate:"2019-02-28T13:35:32Z", GoVersion:"go1.11.5", Compiler:"gc", Platform:"linux/amd64"}

$ docker version
Client:
 Version:           18.09.3
 API version:       1.39
 Go version:        go1.10.6
 Git commit:        142dfce
 Built:             Thu Feb 28 06:08:06 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine
 Engine:
  Version:          18.09.3
  API version:      1.39 (minimum version 1.12)
  Go version:       go1.10.8
  Git commit:       142dfce
  Built:            Thu Feb 28 06:03:18 2019
  OS/Arch:          linux/amd64
  Experimental:     false

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.4", GitCommit:"c27b913fddd1a6c480c229191a087698aa92f0b1", GitTreeState:"clean", BuildDate:"2019-02-28T13:37:52Z", GoVersion:"go1.11.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.4", GitCommit:"c27b913fddd1a6c480c229191a087698aa92f0b1", GitTreeState:"clean", BuildDate:"2019-02-28T13:30:26Z", GoVersion:"go1.11.5", Compiler:"gc", Platform:"linux/amd64"}