[Good First Issue] MonitoringEKS example is using outdated k8s version 1.14
rl-abdulkarim opened this issue · comments
Problem: The MonitoringEKS example is using k8s version 1.14 which is no longer supported by Amazon EKS.
❯ eksctl create cluster --name my-cluster2 --version 1.14
2021-05-05 00:29:51 [ℹ] eksctl version 0.47.0
2021-05-05 00:29:51 [ℹ] using region us-west-2
Error: invalid version, 1.14 is no longer supported, supported values: 1.15, 1.16, 1.17, 1.18, 1.19
see also: https://docs.aws.amazon.com/eks/latest/userguide/kubernetes-versions.html
Suggested Fix: Use k8s version 1.19 (since it's the latest available version on Amazon EKS). This will require
- Fixing the README.md
- Fixing the
beats/*.yaml
files to support new k8s version
I agree with @rl-abdulkarim the EKS example needs updated. I was following the tutorial by Elastic here but had errors such as:
resource mapping not found for name: "metricbeat" namespace: "kube-system" from "examples/MonitoringEKS/beats/metricbeat-kubernetes.yaml": no matches for kind "DaemonSet" in version "extensions/v1beta1"
ensure CRDs are installed first
error validating data: ValidationError(DaemonSet.spec): missing required field "selector" in io.k8s.api.apps.v1.DaemonSetSpec
I had to amend the metricbeat-kubernetes.yaml
file changing outdated api versions:
apiVersion: extensions/v1beta1
apiVersion: apps/v1beta1
apiVersion: rbac.authorization.k8s.io/v1beta1
to
apiVersion: apps/v1
apiVersion: rbac.authorization.k8s.io/v1
Next I had to add spec sections (as below) to the Daemonset and Deployment sections:
spec:
selector:
matchLabels:
k8s-app: metricbeat
Also in the generated secrets.yaml file I had to amend the name:
apiVersion: v1
kind: Secret
metadata:
name: beats-secrets
The generation script creates the name as: metricbeat-secrets
rather than beats-secrets
I agree with @rl-abdulkarim the EKS example needs updated. I was following the tutorial by Elastic here but had errors such as:
resource mapping not found for name: "metricbeat" namespace: "kube-system" from "examples/MonitoringEKS/beats/metricbeat-kubernetes.yaml": no matches for kind "DaemonSet" in version "extensions/v1beta1" ensure CRDs are installed first error validating data: ValidationError(DaemonSet.spec): missing required field "selector" in io.k8s.api.apps.v1.DaemonSetSpec
I had to amend the
metricbeat-kubernetes.yaml
file changing outdated api versions:apiVersion: extensions/v1beta1 apiVersion: apps/v1beta1 apiVersion: rbac.authorization.k8s.io/v1beta1
to
apiVersion: apps/v1 apiVersion: rbac.authorization.k8s.io/v1
Next I had to add spec sections (as below) to the Daemonset and Deployment sections:
spec: selector: matchLabels: k8s-app: metricbeat
Also in the generated secrets.yaml file I had to amend the name:
apiVersion: v1 kind: Secret metadata: name: beats-secrets
The generation script creates the name as:
metricbeat-secrets
rather thanbeats-secrets
Hello @jonnymccullagh,
May you tell me where exactly this should be placed, because i am bit lost.
spec:
selector:
matchLabels:
k8s-app: metricbeat
This is how my Deamonset section looks like:
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: metricbeat
namespace: kube-system
labels:
k8s-app: metricbeat
spec:
template:
metadata:
labels:
k8s-app: metricbeat
spec:
serviceAccountName: metricbeat
terminationGracePeriodSeconds: 30
hostNetwork: true
dnsPolicy: ClusterFirstWithHostNet
containers:
- name: metricbeat
image: docker.elastic.co/beats/metricbeat:7.6.2
args: [
"-c", "/etc/metricbeat.yml",
"-e",
"-system.hostfs=/hostfs",
]
This is the Deployment section:
apiVersion: apps/v1
kind: Deployment
metadata:
name: metricbeat
namespace: kube-system
labels:
k8s-app: metricbeat
spec:
template:
metadata:
labels:
k8s-app: metricbeat
spec:
serviceAccountName: metricbeat
hostNetwork: true
dnsPolicy: ClusterFirstWithHostNet
containers:
- name: metricbeat
image: docker.elastic.co/beats/metricbeat:7.6.2
args: [
"-c", "/etc/metricbeat.yml",
"-e",
]
Thank you in advance!