Kubernetes-Dashboard Redirection is returning an error

[GuitaristForEver]

Hi, I'm getting an error when clicking the kubernetes dashboard icon on the Orchestra Portal.

Kubernetes version: Azure AKS 1.14.3
Dashboard Version : kubernetesui/dashboard:v2.0.0-beta2
Orceastra version : openunison-k8s-login-oidc:1.0.18 tried with 1.0.17 as well.

What is the problem here? Thanks

[Marc Boorshtein]

what do the logs say in the openunison-orchestra pod?

[GuitaristForEver]

search
Search
Logs
Namespace
openunison
Logs from
openunison-orchestra
in
openunison-orchestra-5f4c8666dd-xkk9z

     [127.0.0.1] - [facaf856069f1f621dd6df4017210ed78f742ae15]

[2019-07-29 18:15:08,069][XNIO-1 task-11] INFO AccessLog - [AzSuccess] - k8sIdp - https://127.0.0.1:8443/auth/idp/k8sIdp/.well-known/openid-configuration - uid=Anonymous,o=Tremolo - NONE [127.0.0.1] - [f8e3769c776749483db478fc7696c2ae25c29dde8]
[2019-07-29 18:15:11,758][XNIO-1 task-14] INFO AccessLog - [AzSuccess] - CheckAlive - https://127.0.0.1:8443/check_alive - uid=Anonymous,o=Tremolo -
[127.0.0.1] - [f4095a1d35930631a55269286e709b7649ff9fe58]
[2019-07-29 18:15:12,373][XNIO-1 task-12] INFO AccessLog - [AzSuccess] - k8sIdp - https://aks-devops-login.clipro.tv/auth/idp/k8sIdp/completefed - uid=yuval.gabayx-64-xwsc-sports.com,ou=shadow,o=Tremolo - [10.244.3.22] - [fecbfe251609a5d8e806f29031ac109b1b3fa1ab0]
[2019-07-29 18:15:12,732][XNIO-1 task-3] INFO AccessLog - [Error] - k8s - https://aks-devops-dashboard.clipro.tv/auth/oidc - uid=Anonymous,o=Tremolo - NONE [10.244.3.22] - [f03f250a79b67bf3c9b0398248d61dccc5dae1527]
[2019-07-29 18:15:12,732][XNIO-1 task-3] ERROR ConfigSys - Could not process request
javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) ~[?:1.8.0_212]
at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1946) ~[?:1.8.0_212]
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:316) ~[?:1.8.0_212]
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:310) ~[?:1.8.0_212]
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1639) ~[?:1.8.0_212]
at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:223) ~[?:1.8.0_212]
at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1037) ~[?:1.8.0_212]
at sun.security.ssl.Handshaker.process_record(Handshaker.java:965) ~[?:1.8.0_212]
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1064) ~[?:1.8.0_212]
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1367) ~[?:1.8.0_212]
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1395) ~[?:1.8.0_212]
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1379) ~[?:1.8.0_212]
at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:436) ~[httpclient-4.5.9.jar:4.5.9]
Logs from 7/29/19 6:15 PM to 7/29/19 6:15 PM UTC

[Marc Boorshtein]

Looks like a certificate trust issue. Did you redeploy openunison or did you redeploy the dashboard? If you redeployed the dashboard delete the secret kubernetes-dashboard-certs in the kubernetes-dashboard namespace and add an annotation (it doesn't matter which one) to the orchestra OpenUnison object in the openunison namespace. This will trigger the operator to recreate the dashboard certificates for you.

[GuitaristForEver]

I deleted the cert from the dashboard namespace and created an annotation , I even triggered the check-certs-orchestra job
this is the output of this pod :

not expiring
Key 'unison-ca' finished
Processing key 'kubernetes-dashboard'
Checking if kubernetes secret exists
Key 'kubernetes-dashboard' finished
Processing key 'unison-saml2-rp-sig'
Checking if kubernetes secret exists
Secret exists

still, the cert is not seen in the dashboard and still getting the error...
and getting the same error in the comment above.

BTW, I did deploy the orchestra again after editing some configurations so any extra actions needed as well?

Thanks 👍

[Marc Boorshtein]

Lets go through a checklist. I'm going to assume that the 2.0 dashboard is deployed to kubernetes-dashboard.:

1. in the kubernetes-dashbaord namespace is there a secret called kubernetes-dashboard-certs? Does it have labels like:

labels:
    operated-by: openunison-operator
    tremolo_operator_created: "true"

2. Try killing the kubernetes-dashboard-XXX-XXXX pod and see if the redirect works

3. In the openunison namespace, in the orchestra OpenUnison custom resource:
   a. Look for a kubernetes-dashboard in the keystore, does it look like:

   - create_data:
       ca_cert: true
       delete_pods_labels:
       - k8s-app=kubernetes-dashboard
       key_size: 2048
       secret_info:
         cert_name: dashboard.crt
         key_name: dashboard.key
         type_of_secret: Opaque
       server_name: kubernetes-dashboard.kubernetes-dashboard.svc.cluster.local
       sign_by_k8s_ca: false
       subject_alternative_names: []
       target_namespace: kubernetes-dashboard
     import_into_ks: certificate
     name: kubernetes-dashboard
     replace_if_exists: true
     tls_secret_name: kubernetes-dashboard-certs
   

   b. Under non_secret_data is K8S_DASHBOARD_NAMESPACE set to kubernetes-dashboard?

[GuitaristForEver]

I went step by step with the checklist:

after changing this value : import_into_ks: certificate
the certificate has been created in the kubernetes-dashboard namespace, labels appear to match for yours.

Still, the same error with the same Log on the Orchestra pod
killing the dashboard pod did not help.

[GuitaristForEver]

after contacting Microsoft and with the help of @mlbiam we managed to custom the orchestra image with enterprise idp with. after that I was stumbling with the URL not reaching to the ingress, I reached Microsoft and it seems that the problem was with my AKS kubernetes version. I created a new version of AKS.

you can close this issue now.