How can I import my own certificates?

Question

How can I import my own certificates?

eyanez111 opened this issue 2 years ago · comments

Hello Community,

    I am trying to add to heartbeat (all our infra is running in kubernetes) all our pods that have certs so we can monitor those via https and alert when is time to renew them. To access those via http I need to import certs to the hearbet yaml config. I saw that on the elastic documentation:

https://www.elastic.co/guide/en/beats/heartbeat/7.16/monitor-http-options.html#monitor-http-tls-ssl

type: http
id: my-http-service
name: My HTTP Service
hosts: ["https://myhost:443"]
schedule: '@every 5s'
ssl:
certificate_authorities: ['/etc/ca.crt']
supported_protocols: ["TLSv1.0", "TLSv1.1", "TLSv1.2"]

What I do not find is how to add those certs to the pod? I have the secrets of the certs but I am not sure if I should create a secret for heatbeat? or if I should modify the existing one that I see in the kube-system:

heartbeat-token-lk8sl kubernetes.io/service-account-token 3 2d1h

if I have to create one do you have a process on how by creation it will be called by hearbeat? do I have to use any apiVersion? and if it is done by modifying the heartbeat token do I need to replace the ca.crt that I see there?

Thanks
eyanez111

Dan Roscigno · Answer 1 · Fri Apr 29 2022 05:35:32 GMT+0800 (China Standard Time)

Hi @eyanez111

In your example config above you are saying that the cert is at /etc/ca.crt, so you would mount the secret as a volume at /etc/ca.crt. Take a look at this example, the person who needed the cert was using it to connect to Elasticsearch, but it does not matter. Just create the secret, create the volume mount, and create the volume. I have not added a cert to Heartbeat this way, but I have done this for other software.

https://stackoverflow.com/questions/48757833/how-to-add-self-signed-certificate-in-kubernetes-elastic-search-deployment-using

Francisco Yanez · Answer 2 · Fri Apr 29 2022 07:17:13 GMT+0800 (China Standard Time)

Thanks for the answer. I really appreciate it.

   So this is the example provided by Elastic:

ssl:
certificate_authorities: ['/etc/ca.crt']
supported_protocols: ["TLSv1.0", "TLSv1.1", "TLSv1.2"]

I assume then that based on the link you shared if I already have the cert on an existing secret that the website uses all what I have to do is:

ssl:
certificate_authorities: ['/etc/ca.crt']
supported_protocols: ["TLSv1.0", "TLSv1.1", "TLSv1.2"]

volumeMounts:
- name: cert
  mountPath: /etc/ssl/certs
...

volumes:
- name: cert
  secret:
  secretName: **cert-from-the-website-I-want-to-monitor**
  defaultMode: 0400

or I do not need the part:
ssl:
certificate_authorities: ['/etc/ca.crt']
supported_protocols: ["TLSv1.0", "TLSv1.1", "TLSv1.2"]

Thanks again

Dan Roscigno · Answer 3 · Fri Apr 29 2022 18:39:50 GMT+0800 (China Standard Time)

I think you have it mostly correct, just change this piece:

volumeMounts:
- name: cert
  mountPath: /etc/ssl/certs

to

volumeMounts:
- name: cert
  mountPath: /etc/ca.crt
  subPath: config.json
  readOnly: true

If it does not work, then have a look at the logs and shell into the container and have a look at the file mounted at /etc/ca.crt. It has been a couple years since I worked on Beats in k8s.

Dan Roscigno · Answer 4 · Fri Apr 29 2022 20:11:00 GMT+0800 (China Standard Time)

There is also an example that I wrote years ago that shows how to create a configmap that contains the cert and mount it. The cert in the example is the cert to connect to Elasticsearch, so that is not what you want (I think in your use case you want to give Heartbeat a cert to connect to a service that you are managing, correct?), but the technique is the same.

Francisco Yanez · Answer 5 · Wed May 04 2022 00:43:55 GMT+0800 (China Standard Time)

Hello @DanRoscigno,

     Sorry it did not work I did:

ssl:
certificate_authorities: ['/etc/ca.crt']
supported_protocols: ["TLSv1.0", "TLSv1.1", "TLSv1.2"]

volumeMounts:
- name: cert
  mountPath: /etc/ca.crt
  subPath: config.json
  readOnly: true
...

volumes:
- name: cert
  secret:
  secretName: **cert-from-the-website-I-want-to-monitor**
  defaultMode: 0400

 What goes on "subPath: config.json" do I need to leave it like that or should I actually put a path?

Dan Roscigno · Answer 6 · Wed May 04 2022 00:53:35 GMT+0800 (China Standard Time)

the name of the file that you want is subPath (ca.crt). I find it confusing too.

Francisco Yanez · Answer 7 · Wed May 04 2022 00:57:37 GMT+0800 (China Standard Time)

thanks for answering so fast. So it should be like:

volumeMounts:

name: cert
mountPath: /etc/ca.crt
subPath: config.json ---> name of the path for the ca.crt?
readOnly: true

but how do I point at any path if the cert I have it on a secret?

thanks

Dan Roscigno · Answer 8 · Wed May 04 2022 01:29:03 GMT+0800 (China Standard Time)

three steps:

Create secret
Make a volume named cert
Mount the volume certin avolumeMountalso namedcert. This third step is where you specify the filename, by specifying both the mountPathandsubPath. The mountPathincludes the dir name and filename, and thesubPath` is just the filename.

Francisco Yanez · Answer 9 · Wed May 04 2022 01:40:19 GMT+0800 (China Standard Time)

One question. I do have the secret but is in a different namespace do I have to copy that secret and put it on the same namespace that I am running it? I think that is the main thing why it is failing.

as per what you said then I think is like this right

    ssl:
    certificate_authorities: ['/etc/**SECRETNAME**']
    supported_protocols: ["TLSv1.0", "TLSv1.1", "TLSv1.2"]

    volumeMounts:
    - name: cert
      mountPath: /etc/**SECRETNAME**
      subPath: **SECRETNAME**
      readOnly: true

    volumes:
    - name: cert
      secret:
      secretName: **SECRETNAME**
      defaultMode: 0400

Thanks a lot for all the help

Dan Roscigno · Answer 10 · Wed May 04 2022 01:52:42 GMT+0800 (China Standard Time)

My pleasure!

The secret definitely needs to be in the same namespace unless your pod is running with admin level.

kubectl describe pod and kubectl exec -it demo-pod -- /bin/sh will help you debug it. Open a shell into the pod and look at /etc/ca.crt to see if it contains the right thing, and describe will tell you about the secret and volume.

Francisco Yanez · Answer 11 · Wed May 04 2022 03:17:51 GMT+0800 (China Standard Time)

ah sorry I just talked to DEV and looks like we do not have a hold of the cert. We only have the tls.cert. So there is not a CA involved. On the secret I just have the tls.cert and the tls.key. Is it possible to do it like that:

ssl:
    certificate_authorities: ['/etc/tls.crt']
    supported_protocols: ["TLSv1.0", "TLSv1.1", "TLSv1.2"]

    volumeMounts:
    - name: cert
      mountPath: /etc/tls.crt
      subPath: tls.crt
      readOnly: true

    volumes:
    - name: cert
      secret: kubernetes.io/tls
      secretName: CERT-NAME
      defaultMode: 0400

again deeply thank you as not even Elastic support has been able to point me the right direction.

Dan Roscigno · Answer 12 · Wed May 04 2022 03:20:58 GMT+0800 (China Standard Time)

I would get it working without k8s first, then work on k8s.

Francisco Yanez · Answer 13 · Wed May 04 2022 03:22:44 GMT+0800 (China Standard Time)

sorry but I just have a k8s environment.

Francisco Yanez · Answer 14 · Wed May 04 2022 03:51:27 GMT+0800 (China Standard Time)

ah, I was able to get the ca after talking to devs and IT. So I was ready to proceed but after:

Moving the secret (I added the CA)
Going create the volumes:

        volumeMounts:
        - name: cert
          mountPath: /etc/tls.cert
          readOnly: true
          subPath: tls.crt
      volumes:
      - name: cert
        secret: kubernetes.io/tls
        secretName: NAME
        defaultMode: 0400

This is what I get:
error: error validating "heartbeat-kubernetes-prod-rancher.yaml": error validating data: [ValidationError(Deployment.spec.template.spec.volumes[0]): unknown field "defaultMode" in io.k8s.api.core.v1.Volume, ValidationError(Deployment.spec.template.spec.volumes[0].secret): invalid type for io.k8s.api.core.v1.SecretVolumeSource: got "string", expected "map", ValidationError(Deployment.spec.template.spec.volumes[0]): unknown field "secretName" in io.k8s.api.core.v1.Volume]; if you choose to ignore these errors, turn validation off with --validate=false

so not sure why is not recognizing the secretName... any idea? I think I am very very close!

Dan Roscigno · Answer 15 · Wed May 04 2022 04:16:26 GMT+0800 (China Standard Time)

I don't think secretName belongs in the cert volume. Look at this example, and setup the cert like that. If you need more details on the volumes, etc., have a look at the k8s docs.

Francisco Yanez · Answer 16 · Wed May 04 2022 04:33:04 GMT+0800 (China Standard Time)

I found the error. I was adding what type of secret and secret is just to define that it is a secret what is trying to mount. so I just left it as Secret that worked. I see now the cert in the pod. I think I am quite close.... thanks again for all the help without your assistance I would not have been able to do this.

Dan Roscigno · Answer 17 · Wed May 04 2022 04:42:05 GMT+0800 (China Standard Time)

my pleasure, once you get it all working can you pop the manifest in here for the next person?

Francisco Yanez · Answer 18 · Wed May 04 2022 04:44:28 GMT+0800 (China Standard Time)

for sure, I am still working on it but at least I see the mount and the cert there. So definitely heading the right direction

Francisco Yanez · Answer 19 · Wed May 18 2022 02:23:08 GMT+0800 (China Standard Time)

I have not been able to make it work with the tls but I think this means it will only work with the ca.crt and not with the tls.crt. Still the process what you described @DanRoscigno it was pretty much this:

Create secret

 apiVersion: v1
 kind: Secret
 metadata:
   name: self-signed-certificate-secret
 data:
   ca-certificates.crt: <base 64 encoded string of certificate>

Make a volume named cert:

   volumes: 
        -  
          secret: 
            secretName: NAME FOR YOUR SECRET
            defaultMode: 0400
          name: cert

Mount the volume certin avolumeMountalso namedcert. This third step is where you specify the filename, by specifying both the mountPathandsubPath. The mountPathincludes the dir name and filename, and thesubPath` is just the filename.
```
   volumeMounts: 
     - 
       mountPath: /etc/ca.crt
       name: cert
       readOnly: true
       subPath: ca.crt
```
Last step for heatbeat you need to add this on the hearbeat.yaml part:

    - type: http
      id: service
      name:  NAME
      hosts: ["https://YOUR-URL"]
      schedule: '@every 5s'
      ssl:
        certificate_authorities: ['/etc/ca.crt']
        supported_protocols: ["TLSv1.0", "TLSv1.1", "TLSv1.2"]

That is how you add a secret based on a CA to heartbeat thanks again @DanRoscigno