AKS k8s giving authorization error ERROR K8sWatcher - Could not get authentication token
sharmavijay86 opened this issue · comments
I am configuring openunison-k8s-login-github with aks cluster.
error which i am getting is after github auth it gives
Not Authorized
You are not authorized for failed authentication. If you feel you received this message in error, please contact your system administrator or help desk.
logs shows --
[2021-04-20 04:44:10,461][XNIO-1 task-2] INFO AccessLog - [AzSuccess] - CheckAlive - https://127.0.0.1:8443/check_alive - uid=Anonymous,o=Tremolo -
[127.0.0.1] - [f173d7ea04088bb727f70089ead2064aa549784ff]
[2021-04-20 04:44:11,546][XNIO-1 task-2] INFO AccessLog - [AzSuccess] - k8sIdp - https://127.0.0.1:8443/auth/idp/k8sIdp/.well-known/openid-configuration - uid=Anonymous,o=Tremolo - NONE [127.0.0.1] - [fffbc01d38681028c78aa4ded8de22c36e6255bf4]
[2021-04-20 04:44:20,472][XNIO-1 task-2] INFO AccessLog - [AzSuccess] - CheckAlive - https://127.0.0.1:8443/check_alive - uid=Anonymous,o=Tremolo -
[127.0.0.1] - [f9bd32b7868b1a1cb2233cba9093d66d9a8577e9a]
[2021-04-20 04:44:21,545][XNIO-1 task-2] INFO AccessLog - [AzSuccess] - k8sIdp - https://127.0.0.1:8443/auth/idp/k8sIdp/.well-known/openid-configuration - uid=Anonymous,o=Tremolo - NONE [127.0.0.1] - [fba23bb85afb34a4f463381000deba902547ba7a1]
[2021-04-20 04:44:30,461][XNIO-1 task-2] INFO AccessLog - [AzSuccess] - CheckAlive - https://127.0.0.1:8443/check_alive - uid=Anonymous,o=Tremolo -
[127.0.0.1] - [ffa59609315ef90944e6b99e76acfebeb040db16f]
[2021-04-20 04:44:31,548][XNIO-1 task-2] INFO AccessLog - [AzSuccess] - k8sIdp - https://127.0.0.1:8443/auth/idp/k8sIdp/.well-known/openid-configuration - uid=Anonymous,o=Tremolo - NONE [127.0.0.1] - [f752edc680f8a8c20fcc02c97e0088f0b636925e6]
[2021-04-20 04:44:36,582][Thread-9] ERROR K8sWatcher - Could not get authentication token
javax.net.ssl.SSLException: Connection reset
at sun.security.ssl.Alert.createSSLException(Alert.java:127) ~[?:?]
at sun.security.ssl.TransportContext.fatal(TransportContext.java:349) ~[?:?]
at sun.security.ssl.TransportContext.fatal(TransportContext.java:292) ~[?:?]
at sun.security.ssl.TransportContext.fatal(TransportContext.java:287) ~[?:?]
at sun.security.ssl.SSLSocketImpl.handleException(SSLSocketImpl.java:1581) ~[?:?]
at sun.security.ssl.SSLSocketImpl$AppInputStream.read(SSLSocketImpl.java:979) ~[?:?]
at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137) ~[httpcore-4.4.14.jar:4.4.14]
at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153) ~[httpcore-4.4.14.jar:4.4.14]
at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:280) ~[httpcore-4.4.14.jar:4.4.14]
at org.apache.http.impl.io.ChunkedInputStream.getChunkSize(ChunkedInputStream.java:261) ~[httpcore-4.4.14.jar:4.4.14]
at org.apache.http.impl.io.ChunkedInputStream.nextChunk(ChunkedInputStream.java:222) ~[httpcore-4.4.14.jar:4.4.14]
at org.apache.http.impl.io.ChunkedInputStream.read(ChunkedInputStream.java:183) ~[httpcore-4.4.14.jar:4.4.14]
at org.apache.http.conn.EofSensorInputStream.read(EofSensorInputStream.java:135) ~[httpclient-4.5.9.jar:4.5.9]
at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284) ~[?:?]
at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326) ~[?:?]
at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178) ~[?:?]
at java.io.InputStreamReader.read(InputStreamReader.java:181) ~[?:?]
at java.io.BufferedReader.fill(BufferedReader.java:161) ~[?:?]
at java.io.BufferedReader.readLine(BufferedReader.java:326) ~[?:?]
at java.io.BufferedReader.readLine(BufferedReader.java:392) ~[?:?]
at com.tremolosecurity.k8s.watch.K8sWatcher.run(K8sWatcher.java:205) [unison-applications-k8s-1.0.22.jar:?]
at java.lang.Thread.run(Thread.java:834) [?:?]
Suppressed: java.net.SocketException: Broken pipe (Write failed)
at java.net.SocketOutputStream.socketWrite0(Native Method) ~[?:?]
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:110) ~[?:?]
at java.net.SocketOutputStream.write(SocketOutputStream.java:150) ~[?:?]
at sun.security.ssl.SSLSocketOutputRecord.encodeAlert(SSLSocketOutputRecord.java:81) ~[?:?]
at sun.security.ssl.TransportContext.fatal(TransportContext.java:380) ~[?:?]
at sun.security.ssl.TransportContext.fatal(TransportContext.java:292) ~[?:?]
at sun.security.ssl.TransportContext.fatal(TransportContext.java:287) ~[?:?]
at sun.security.ssl.SSLSocketImpl.handleException(SSLSocketImpl.java:1581) ~[?:?]
at sun.security.ssl.SSLSocketImpl$AppInputStream.read(SSLSocketImpl.java:979) ~[?:?]
at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137) ~[httpcore-4.4.14.jar:4.4.14]
at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153) ~[httpcore-4.4.14.jar:4.4.14]
at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:280) ~[httpcore-4.4.14.jar:4.4.14]
at org.apache.http.impl.io.ChunkedInputStream.getChunkSize(ChunkedInputStream.java:261) ~[httpcore-4.4.14.jar:4.4.14]
at org.apache.http.impl.io.ChunkedInputStream.nextChunk(ChunkedInputStream.java:222) ~[httpcore-4.4.14.jar:4.4.14]
at org.apache.http.impl.io.ChunkedInputStream.read(ChunkedInputStream.java:183) ~[httpcore-4.4.14.jar:4.4.14]
at org.apache.http.conn.EofSensorInputStream.read(EofSensorInputStream.java:135) ~[httpclient-4.5.9.jar:4.5.9]
at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284) ~[?:?]
at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326) ~[?:?]
at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178) ~[?:?]
at java.io.InputStreamReader.read(InputStreamReader.java:181) ~[?:?]
at java.io.BufferedReader.fill(BufferedReader.java:161) ~[?:?]
at java.io.BufferedReader.readLine(BufferedReader.java:326) ~[?:?]
at java.io.BufferedReader.readLine(BufferedReader.java:392) ~[?:?]
at com.tremolosecurity.k8s.watch.K8sWatcher.run(K8sWatcher.java:205) [unison-applications-k8s-1.0.22.jar:?]
at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:186) ~[?:?]
at java.net.SocketInputStream.read(SocketInputStream.java:140) ~[?:?]
at sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:478) ~[?:?]
at sun.security.ssl.SSLSocketInputRecord.readHeader(SSLSocketInputRecord.java:472) ~[?:?]
at sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(SSLSocketInputRecord.java:70) ~[?:?]
at sun.security.ssl.SSLSocketImpl.readApplicationRecord(SSLSocketImpl.java:1354) ~[?:?]
at sun.security.ssl.SSLSocketImpl$AppInputStream.read(SSLSocketImpl.java:963) ~[?:?]
... 16 more
[2021-04-20 04:44:40,462][XNIO-1 task-2] INFO AccessLog - [AzSuccess] - CheckAlive - https://127.0.0.1:8443/check_alive - uid=Anonymous,o=Tremolo -
[127.0.0.1] - [ff6fccab3484cb5d06689c5b5758a98c9dc26ab1c]
[2021-04-20 04:44:41,539][XNIO-1 task-2] INFO AccessLog - [AzSuccess] - k8sIdp - https://127.0.0.1:8443/auth/idp/k8sIdp/.well-known/openid-configuration - uid=Anonymous,o=Tremolo - NONE [127.0.0.1] - [f9edb5aeda6319fd9d5356341763e0d67b811939f]
[2021-04-20 04:44:48,633][XNIO-1 task-2] INFO GithubAuthMech - login type - 'java.lang.String'
[2021-04-20 04:44:48,633][XNIO-1 task-2] INFO GithubAuthMech - id type - 'java.lang.Long'
[2021-04-20 04:44:48,634][XNIO-1 task-2] INFO GithubAuthMech - node_id type - 'java.lang.String'
[2021-04-20 04:44:48,635][XNIO-1 task-2] INFO GithubAuthMech - avatar_url type - 'java.lang.String'
[2021-04-20 04:44:48,635][XNIO-1 task-2] INFO GithubAuthMech - gravatar_id type - 'java.lang.String'
[2021-04-20 04:44:48,635][XNIO-1 task-2] INFO GithubAuthMech - url type - 'java.lang.String'
[2021-04-20 04:44:48,637][XNIO-1 task-2] INFO GithubAuthMech - html_url type - 'java.lang.String'
[2021-04-20 04:44:48,637][XNIO-1 task-2] INFO GithubAuthMech - followers_url type - 'java.lang.String'
[2021-04-20 04:44:48,643][XNIO-1 task-2] INFO GithubAuthMech - following_url type - 'java.lang.String'
[2021-04-20 04:44:48,644][XNIO-1 task-2] INFO GithubAuthMech - gists_url type - 'java.lang.String'
[2021-04-20 04:44:48,647][XNIO-1 task-2] INFO GithubAuthMech - starred_url type - 'java.lang.String'
[2021-04-20 04:44:48,649][XNIO-1 task-2] INFO GithubAuthMech - subscriptions_url type - 'java.lang.String'
[2021-04-20 04:44:48,649][XNIO-1 task-2] INFO GithubAuthMech - organizations_url type - 'java.lang.String'
[2021-04-20 04:44:48,649][XNIO-1 task-2] INFO GithubAuthMech - repos_url type - 'java.lang.String'
[2021-04-20 04:44:48,649][XNIO-1 task-2] INFO GithubAuthMech - events_url type - 'java.lang.String'
[2021-04-20 04:44:48,651][XNIO-1 task-2] INFO GithubAuthMech - received_events_url type - 'java.lang.String'
[2021-04-20 04:44:48,651][XNIO-1 task-2] INFO GithubAuthMech - type type - 'java.lang.String'
[2021-04-20 04:44:48,653][XNIO-1 task-2] INFO GithubAuthMech - site_admin type - 'java.lang.Boolean'
[2021-04-20 04:44:48,653][XNIO-1 task-2] INFO GithubAuthMech - name type - 'java.lang.String'
[2021-04-20 04:44:48,653][XNIO-1 task-2] INFO GithubAuthMech - company type - 'java.lang.String'
[2021-04-20 04:44:48,654][XNIO-1 task-2] INFO GithubAuthMech - blog type - 'java.lang.String'
[2021-04-20 04:44:48,655][XNIO-1 task-2] INFO GithubAuthMech - location type - 'java.lang.String'
[2021-04-20 04:44:48,655][XNIO-1 task-2] INFO GithubAuthMech - email type - 'java.lang.String'
[2021-04-20 04:44:48,656][XNIO-1 task-2] INFO GithubAuthMech - hireable type - 'java.lang.Boolean'
[2021-04-20 04:44:48,657][XNIO-1 task-2] INFO GithubAuthMech - bio type - 'java.lang.String'
[2021-04-20 04:44:48,657][XNIO-1 task-2] INFO GithubAuthMech - public_repos type - 'java.lang.Long'
[2021-04-20 04:44:48,658][XNIO-1 task-2] INFO GithubAuthMech - public_gists type - 'java.lang.Long'
[2021-04-20 04:44:48,658][XNIO-1 task-2] INFO GithubAuthMech - followers type - 'java.lang.Long'
[2021-04-20 04:44:48,659][XNIO-1 task-2] INFO GithubAuthMech - following type - 'java.lang.Long'
[2021-04-20 04:44:48,659][XNIO-1 task-2] INFO GithubAuthMech - created_at type - 'java.lang.String'
[2021-04-20 04:44:48,659][XNIO-1 task-2] INFO GithubAuthMech - updated_at type - 'java.lang.String'
[2021-04-20 04:44:48,660][XNIO-1 task-2] INFO GithubAuthMech - mail type - 'java.lang.String'
[2021-04-20 04:44:48,986][XNIO-1 task-2] INFO AccessLog - [AuFail] - scale - https://k8sou.aks.mevijay.site/auth/github - cn=none - enterprise_idp [10.244.1.9] - [f1887cca5b9b1024b6370237a638205e005dca60e]
[2021-04-20 04:44:50,251][XNIO-1 task-2] INFO AccessLog - [AzSuccess] - anonfiles - https://k8sou.aks.mevijay.site/favicon.ico - uid=Anonymous,o=Tremolo -
[10.244.1.9] - [f05c035239c45d34a429c22367b898acc84ebdf16]
[2021-04-20 04:44:50,455][XNIO-1 task-2] INFO AccessLog - [AzSuccess] - CheckAlive - https://127.0.0.1:8443/check_alive - uid=Anonymous,o=Tremolo -
[127.0.0.1] - [fd4e46ba4aa9d0ebf3291d8953b9f093cc656ffc9]
[2021-04-20 04:44:51,535][XNIO-1 task-2] INFO AccessLog - [AzSuccess] - k8sIdp - https://127.0.0.1:8443/auth/idp/k8sIdp/.well-known/openid-configuration - uid=Anonymous,o=Tremolo - NONE [127.0.0.1] - [f748850f949865a6c7eab0ed36437a5a9759196e7]
[2021-04-20 04:45:00,103][Thread-10] WARN SessionManagerImpl - Clearing 0 sessions
[2021-04-20 04:45:00,459][XNIO-1 task-2] INFO AccessLog - [AzSuccess] - CheckAlive - https://127.0.0.1:8443/check_alive - uid=Anonymous,o=Tremolo -
[127.0.0.1] - [ffb6b7326ca278deb203fa3cabe1096e4e124dae4]
[2021-04-20 04:45:01,554][XNIO-1 task-2] INFO AccessLog - [AzSuccess] - k8sIdp - https://127.0.0.1:8443/auth/idp/k8sIdp/.well-known/openid-configuration - uid=Anonymous,o=Tremolo - NONE [127.0.0.1] - [fb3bfe9d319c8b05cf38cb7e6b9c9bd130231a223]```
This error is common in AKS + EKS + onprem kubernetes vanilla as well.
pls find new logs of orchestra pod here -
`[2021-04-20 10:08:46,972][Thread-10] WARN SessionManagerImpl - Clearing 0 sessions
[2021-04-20 10:08:47,249][XNIO-1 task-1] INFO AccessLog - [AzSuccess] - CheckAlive - https://127.0.0.1:8443/check_alive - uid=Anonymous,o=Tremolo -
[127.0.0.1] - [ff3a68c3549a24a528c2fd8ff0e57743fa0a7e5ac]
[2021-04-20 10:08:47,441][XNIO-1 task-1] INFO AccessLog - [AzSuccess] - k8sIdp - https://127.0.0.1:8443/auth/idp/k8sIdp/.well-known/openid-configuration - uid=Anonymous,o=Tremolo - NONE [127.0.0.1] - [fb8bb11980c102cb83457017a3a584b44eb9c1bc6]
[2021-04-20 10:08:57,253][XNIO-1 task-1] INFO AccessLog - [AzSuccess] - CheckAlive - https://127.0.0.1:8443/check_alive - uid=Anonymous,o=Tremolo -
[127.0.0.1] - [fd624e6850777693d931a949cad1a93cb9fd4620f]
[2021-04-20 10:08:57,440][XNIO-1 task-1] INFO AccessLog - [AzSuccess] - k8sIdp - https://127.0.0.1:8443/auth/idp/k8sIdp/.well-known/openid-configuration - uid=Anonymous,o=Tremolo - NONE [127.0.0.1] - [f742c5150023d20f6e90a98c8940a919324f892d7]
[2021-04-20 10:09:07,246][XNIO-1 task-1] INFO AccessLog - [AzSuccess] - CheckAlive - https://127.0.0.1:8443/check_alive - uid=Anonymous,o=Tremolo -
[127.0.0.1] - [f72b4076e4ab4eab40459f01971dd5e326d9b6a31]
[2021-04-20 10:09:07,408][XNIO-1 task-1] INFO AccessLog - [AzSuccess] - k8sIdp - https://127.0.0.1:8443/auth/idp/k8sIdp/.well-known/openid-configuration - uid=Anonymous,o=Tremolo - NONE [127.0.0.1] - [f59affab355a94d5f3af1512f18afffba12493f5b]
[2021-04-20 10:09:17,249][XNIO-1 task-1] INFO AccessLog - [AzSuccess] - CheckAlive - https://127.0.0.1:8443/check_alive - uid=Anonymous,o=Tremolo -
[127.0.0.1] - [f93eecb0c3253957b09c7f6b4db90bbf90cb1bb14]
[2021-04-20 10:09:17,412][XNIO-1 task-1] INFO AccessLog - [AzSuccess] - k8sIdp - https://127.0.0.1:8443/auth/idp/k8sIdp/.well-known/openid-configuration - uid=Anonymous,o=Tremolo - NONE [127.0.0.1] - [f50a7624012cfc4ace9ffb9d7ce4dd0f80ec50836]
[2021-04-20 10:09:24,652][XNIO-1 task-1] INFO GithubAuthMech - login type - 'java.lang.String'
[2021-04-20 10:09:24,653][XNIO-1 task-1] INFO GithubAuthMech - id type - 'java.lang.Long'
[2021-04-20 10:09:24,653][XNIO-1 task-1] INFO GithubAuthMech - node_id type - 'java.lang.String'
[2021-04-20 10:09:24,653][XNIO-1 task-1] INFO GithubAuthMech - avatar_url type - 'java.lang.String'
[2021-04-20 10:09:24,653][XNIO-1 task-1] INFO GithubAuthMech - gravatar_id type - 'java.lang.String'
[2021-04-20 10:09:24,653][XNIO-1 task-1] INFO GithubAuthMech - url type - 'java.lang.String'
[2021-04-20 10:09:24,653][XNIO-1 task-1] INFO GithubAuthMech - html_url type - 'java.lang.String'
[2021-04-20 10:09:24,653][XNIO-1 task-1] INFO GithubAuthMech - followers_url type - 'java.lang.String'
[2021-04-20 10:09:24,653][XNIO-1 task-1] INFO GithubAuthMech - following_url type - 'java.lang.String'
[2021-04-20 10:09:24,653][XNIO-1 task-1] INFO GithubAuthMech - gists_url type - 'java.lang.String'
[2021-04-20 10:09:24,653][XNIO-1 task-1] INFO GithubAuthMech - starred_url type - 'java.lang.String'
[2021-04-20 10:09:24,653][XNIO-1 task-1] INFO GithubAuthMech - subscriptions_url type - 'java.lang.String'
[2021-04-20 10:09:24,653][XNIO-1 task-1] INFO GithubAuthMech - organizations_url type - 'java.lang.String'
[2021-04-20 10:09:24,653][XNIO-1 task-1] INFO GithubAuthMech - repos_url type - 'java.lang.String'
[2021-04-20 10:09:24,653][XNIO-1 task-1] INFO GithubAuthMech - events_url type - 'java.lang.String'
[2021-04-20 10:09:24,653][XNIO-1 task-1] INFO GithubAuthMech - received_events_url type - 'java.lang.String'
[2021-04-20 10:09:24,653][XNIO-1 task-1] INFO GithubAuthMech - type type - 'java.lang.String'
[2021-04-20 10:09:24,653][XNIO-1 task-1] INFO GithubAuthMech - site_admin type - 'java.lang.Boolean'
[2021-04-20 10:09:24,653][XNIO-1 task-1] INFO GithubAuthMech - name type - 'java.lang.String'
[2021-04-20 10:09:24,653][XNIO-1 task-1] INFO GithubAuthMech - company type - 'java.lang.String'
[2021-04-20 10:09:24,653][XNIO-1 task-1] INFO GithubAuthMech - blog type - 'java.lang.String'
[2021-04-20 10:09:24,653][XNIO-1 task-1] INFO GithubAuthMech - location type - 'java.lang.String'
[2021-04-20 10:09:24,653][XNIO-1 task-1] INFO GithubAuthMech - email type - 'java.lang.String'
[2021-04-20 10:09:24,653][XNIO-1 task-1] INFO GithubAuthMech - hireable type - 'java.lang.Boolean'
[2021-04-20 10:09:24,653][XNIO-1 task-1] INFO GithubAuthMech - bio type - 'java.lang.String'
[2021-04-20 10:09:24,653][XNIO-1 task-1] INFO GithubAuthMech - public_repos type - 'java.lang.Long'
[2021-04-20 10:09:24,654][XNIO-1 task-1] INFO GithubAuthMech - public_gists type - 'java.lang.Long'
[2021-04-20 10:09:24,654][XNIO-1 task-1] INFO GithubAuthMech - followers type - 'java.lang.Long'
[2021-04-20 10:09:24,654][XNIO-1 task-1] INFO GithubAuthMech - following type - 'java.lang.Long'
[2021-04-20 10:09:24,654][XNIO-1 task-1] INFO GithubAuthMech - created_at type - 'java.lang.String'
[2021-04-20 10:09:24,654][XNIO-1 task-1] INFO GithubAuthMech - updated_at type - 'java.lang.String'
[2021-04-20 10:09:24,654][XNIO-1 task-1] INFO GithubAuthMech - mail type - 'java.lang.String'
[2021-04-20 10:09:25,578][XNIO-1 task-1] INFO AccessLog - [AuFail] - scale - https://k8sou.k8s.mylab.local/auth/github - cn=none - enterprise_idp [10.46.0.10] - [f097ee9e1796de03b03aa128d091c5de4abe899c6]
[2021-04-20 10:09:26,036][XNIO-1 task-3] INFO AccessLog - [AzSuccess] - anonfiles - https://k8sou.k8s.mylab.local/favicon.ico - uid=Anonymous,o=Tremolo -
[10.46.0.10] - [f096a2a4e85cb0f38d1ad88aa941dc5b7173e74cb]
[2021-04-20 10:09:27,265][XNIO-1 task-3] INFO AccessLog - [AzSuccess] - CheckAlive - https://127.0.0.1:8443/check_alive - uid=Anonymous,o=Tremolo -
[127.0.0.1] - [fa7c9e58dfd32fc07380400897f5f2d4ac464f5dc]
[2021-04-20 10:09:27,432][XNIO-1 task-3] INFO AccessLog - [AzSuccess] - k8sIdp - https://127.0.0.1:8443/auth/idp/k8sIdp/.well-known/openid-configuration - uid=Anonymous,o=Tremolo - NONE [127.0.0.1] - [ff6824f6b24e6cdc2f81f7364e6f145582fdc7c8f]
[2021-04-20 10:09:37,291][XNIO-1 task-3] INFO AccessLog - [AzSuccess] - CheckAlive - https://127.0.0.1:8443/check_alive - uid=Anonymous,o=Tremolo -
[127.0.0.1] - [f87af5ae5e52115df5d6d9ba4d66767e747974986]
[2021-04-20 10:09:37,456][XNIO-1 task-3] INFO AccessLog - [AzSuccess] - k8sIdp - https://127.0.0.1:8443/auth/idp/k8sIdp/.well-known/openid-configuration - uid=Anonymous,o=Tremolo - NONE [127.0.0.1] - [f1738bc734ff6174f7793a5defef5e5ed9c5745b9]
`
This section may need attention -
INFO AccessLog - [AuFail] - scale - https://k8sou.k8s.mylab.local/auth/github - cn=none - enterprise_idp [10.46.0.10] - [f097ee9e1796de03b03aa128d091c5de4abe899c6] [2021-04-20 10:09:26,036][XNIO-1 task-3]
For the excess log data - TremoloSecurity/OpenUnison#533
I'm not seeing any issues with login on on-prem or EKS. Checking AKS now
What does your helm values.yaml look like?
Hi,
This is my value.yaml.
network:
openunison_host: "k8sou.k8s.mylab.local"
dashboard_host: "k8sdb.k8s.mylab.local"
api_server_host: "k8smaster.mylab.local"
session_inactivity_timeout_seconds: 900
k8s_url: "https://k8smaster.mylab.local:6443"
createIngressCertificate: true
ingress_type: nginx
ingress_annotations:
kubernetes.io/ingress.class: nginx
cert-manager.io/cluster-issuer: letsencrypt
cert_template:
ou: "kubernetes"
o: "MyOrg"
l: "aks cluster inc"
st: "Maharashtra"
c: "IN"
image: "docker.io/tremolosecurity/openunison-k8s-login-github:latest"
myvd_config_path: "WEB-INF/myvd.conf"
k8s_cluster_name: kubernetes
enable_impersonation: true
dashboard:
namespace: "kubernetes-dashboard"
cert_name: "kubernetes-dashboard-certs"
label: "k8s-app=kubernetes-dashboard"
service_name: kubernetes-dashboard
certs:
use_k8s_cm: false
trusted_certs: []
monitoring:
prometheus_service_account: system:serviceaccount:monitoring:prometheus-k8s
github:
client_id: 16a0xxxxxxxxxxxxxxx
teams: admin/
impersonation:
use_jetstack: false
jetstack_oidc_proxy_image: quay.io/jetstack/kube-oidc-proxy:v0.3.0
explicit_certificate_trust: true
ca_secret_name: ou-tls-secret
network_policies:
enabled: false
ingress:
enabled: true
labels:
app.kubernetes.io/name: ingress-nginx
monitoring:
enabled: true
labels:
app.kubernetes.io/name: monitoring
apiserver:
enabled: false
labels:
app.kubernetes.io/name: kube-system
services:
enable_tokenrequest: false
token_request_audience: api
token_request_expiration_seconds: 600
node_selectors: []
pullSecret: ""
openunison:
replicas: 1
non_secret_data: {}
secrets: []
There are two items:
createIngressCertificate: true
You're using cert-manager to set your cert so this should be false, i don't think it's your root cause but it could cause issues
teams: admin/
The teams configuration option should be in the form of Organization/team. I think this is the root cause of your issue. Change it to be Organization/team and should work.
Thanks @mlbiam it is working. You were spot on. but kubectl now returns this-
error: You must be logged in to the server (Unauthorized)
However i am running get po just after adding kubeconfig. RBAC also have done with.
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: github-cluster-admins
subjects:
- kind: Group
name: myorg/myteam
roleRef:
kind: ClusterRole
name: cluster-admin
apiGroup: rbac.authorization.k8s.io
- Does the dashboard work? If you're getting items (namespaces, pods, etc) back then everything should be OK on the API server side.
- Are you getting your token using the kubectl oulogin plugin or directly via the tokens screen?
- After getting your kubeconfig setup, can you run
kubectl get ns --v=11?
Dashboard works till login. Namespaces not displaying. bellow is the error log of orchestra pod. This is on AKS cluster and i am using impersonate kube-oidc-proxy.
Values.yaml
network:
openunison_host: "k8sou.aks.xxxxx.site"
dashboard_host: "k8sdb.aks.xxxxxsite"
api_server_host: "myaksclust-myresourcegroup-xxxxxxxxxxxxx.hcp.eastus.azmk8s.io"
session_inactivity_timeout_seconds: 9000
k8s_url: "https://myaksclust-myresourcegroup-xxxxxxxxxxx.hcp.eastus.azmk8s.io:443"
createIngressCertificate: false
ingress_type: nginx
ingress_annotations:
kubernetes.io/ingress.class: nginx
cert-manager.io/cluster-issuer: letsencrypt
cert_template:
ou: "kubernetes"
o: "MyOrg"
l: "aks cluster inc"
st: "Maharashtra"
c: "IN"
image: "docker.io/tremolosecurity/openunison-k8s-login-github:latest"
myvd_config_path: "WEB-INF/myvd.conf"
k8s_cluster_name: myAKSCluster
enable_impersonation: true
dashboard:
namespace: "kubernetes-dashboard"
cert_name: "kubernetes-dashboard-certs"
label: "k8s-app=kubernetes-dashboard"
service_name: kubernetes-dashboard
certs:
use_k8s_cm: false
trusted_certs: []
monitoring:
prometheus_service_account: system:serviceaccount:monitoring:prometheus-k8s
github:
client_id: 16a0xxxxxxxxx
teams: cmyorg/k8sadmin
impersonation:
use_jetstack: true
jetstack_oidc_proxy_image: quay.io/jetstack/kube-oidc-proxy:v0.3.0
explicit_certificate_trust: false
ca_secret_name: ou-tls-secret
network_policies:
enabled: false
ingress:
enabled: true
labels:
app.kubernetes.io/name: ingress-nginx
monitoring:
enabled: true
labels:
app.kubernetes.io/name: monitoring
apiserver:
enabled: false
labels:
app.kubernetes.io/name: kube-system
services:
enable_tokenrequest: false
token_request_audience: api
token_request_expiration_seconds: 900
node_selectors: []
pullSecret: ""
openunison:
replicas: 1
non_secret_data: {}
secrets: []
logs
[10.244.1.9] - [f567855fa72e2b021070274c4f0653a39b61aaf96]
[2021-04-22 15:40:50,211][XNIO-1 task-3] INFO AccessLog - [AzSuccess] - CheckAlive - https://127.0.0.1:8443/check_alive - uid=Anonymous,o=Tremolo -
[127.0.0.1] - [f3b1564d1d17141598eff52a3cfc98b6fe7a224a9]
[2021-04-22 15:40:50,365][XNIO-1 task-3] INFO AccessLog - [AzSuccess] - k8sIdp - https://127.0.0.1:8443/auth/idp/k8sIdp/.well-known/openid-configuration - uid=Anonymous,o=Tremolo - NONE [127.0.0.1] - [f99558d632e6026f0f56beb9a5f22bc71dc22b89e]
[2021-04-22 15:40:57,848][XNIO-1 task-3] INFO AccessLog - [AzSuccess] - ScaleCheckSession - https://k8sou.aks.mevijay.site/scale/sessioncheck - uid=Anonymous,o=Tremolo -
[10.244.1.9] - [f567855fa72e2b021070274c4f0653a39b61aaf96]
[2021-04-22 15:40:58,093][Thread-9] ERROR K8sWatcher - Could not get authentication token
javax.net.ssl.SSLException: Connection reset
at sun.security.ssl.Alert.createSSLException(Alert.java:127) ~[?:?]
at sun.security.ssl.TransportContext.fatal(TransportContext.java:349) ~[?:?]
at sun.security.ssl.TransportContext.fatal(TransportContext.java:292) ~[?:?]
at sun.security.ssl.TransportContext.fatal(TransportContext.java:287) ~[?:?]
at sun.security.ssl.SSLSocketImpl.handleException(SSLSocketImpl.java:1581) ~[?:?]
at sun.security.ssl.SSLSocketImpl$AppInputStream.read(SSLSocketImpl.java:979) ~[?:?]
at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137) ~[httpcore-4.4.14.jar:4.4.14]
at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153) ~[httpcore-4.4.14.jar:4.4.14]
at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:280) ~[httpcore-4.4.14.jar:4.4.14]
at org.apache.http.impl.io.ChunkedInputStream.getChunkSize(ChunkedInputStream.java:261) ~[httpcore-4.4.14.jar:4.4.14]
at org.apache.http.impl.io.ChunkedInputStream.nextChunk(ChunkedInputStream.java:222) ~[httpcore-4.4.14.jar:4.4.14]
at org.apache.http.impl.io.ChunkedInputStream.read(ChunkedInputStream.java:183) ~[httpcore-4.4.14.jar:4.4.14]
at org.apache.http.conn.EofSensorInputStream.read(EofSensorInputStream.java:135) ~[httpclient-4.5.9.jar:4.5.9]
at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284) ~[?:?]
at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326) ~[?:?]
at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178) ~[?:?]
at java.io.InputStreamReader.read(InputStreamReader.java:181) ~[?:?]
at java.io.BufferedReader.fill(BufferedReader.java:161) ~[?:?]
at java.io.BufferedReader.readLine(BufferedReader.java:326) ~[?:?]
at java.io.BufferedReader.readLine(BufferedReader.java:392) ~[?:?]
at com.tremolosecurity.k8s.watch.K8sWatcher.run(K8sWatcher.java:205) [unison-applications-k8s-1.0.22.jar:?]
at java.lang.Thread.run(Thread.java:834) [?:?]
Suppressed: java.net.SocketException: Broken pipe (Write failed)
at java.net.SocketOutputStream.socketWrite0(Native Method) ~[?:?]
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:110) ~[?:?]
at java.net.SocketOutputStream.write(SocketOutputStream.java:150) ~[?:?]
at sun.security.ssl.SSLSocketOutputRecord.encodeAlert(SSLSocketOutputRecord.java:81) ~[?:?]
at sun.security.ssl.TransportContext.fatal(TransportContext.java:380) ~[?:?]
at sun.security.ssl.TransportContext.fatal(TransportContext.java:292) ~[?:?]
at sun.security.ssl.TransportContext.fatal(TransportContext.java:287) ~[?:?]
at sun.security.ssl.SSLSocketImpl.handleException(SSLSocketImpl.java:1581) ~[?:?]
at sun.security.ssl.SSLSocketImpl$AppInputStream.read(SSLSocketImpl.java:979) ~[?:?]
at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137) ~[httpcore-4.4.14.jar:4.4.14]
at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153) ~[httpcore-4.4.14.jar:4.4.14]
at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:280) ~[httpcore-4.4.14.jar:4.4.14]
at org.apache.http.impl.io.ChunkedInputStream.getChunkSize(ChunkedInputStream.java:261) ~[httpcore-4.4.14.jar:4.4.14]
at org.apache.http.impl.io.ChunkedInputStream.nextChunk(ChunkedInputStream.java:222) ~[httpcore-4.4.14.jar:4.4.14]
at org.apache.http.impl.io.ChunkedInputStream.read(ChunkedInputStream.java:183) ~[httpcore-4.4.14.jar:4.4.14]
at org.apache.http.conn.EofSensorInputStream.read(EofSensorInputStream.java:135) ~[httpclient-4.5.9.jar:4.5.9]
at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284) ~[?:?]
at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326) ~[?:?]
at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178) ~[?:?]
at java.io.InputStreamReader.read(InputStreamReader.java:181) ~[?:?]
at java.io.BufferedReader.fill(BufferedReader.java:161) ~[?:?]
at java.io.BufferedReader.readLine(BufferedReader.java:326) ~[?:?]
at java.io.BufferedReader.readLine(BufferedReader.java:392) ~[?:?]
at com.tremolosecurity.k8s.watch.K8sWatcher.run(K8sWatcher.java:205) [unison-applications-k8s-1.0.22.jar:?]
at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:186) ~[?:?]
at java.net.SocketInputStream.read(SocketInputStream.java:140) ~[?:?]
at sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:478) ~[?:?]
at sun.security.ssl.SSLSocketInputRecord.readHeader(SSLSocketInputRecord.java:472) ~[?:?]
at sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(SSLSocketInputRecord.java:70) ~[?:?]
at sun.security.ssl.SSLSocketImpl.readApplicationRecord(SSLSocketImpl.java:1354) ~[?:?]
at sun.security.ssl.SSLSocketImpl$AppInputStream.read(SSLSocketImpl.java:963) ~[?:?]
... 16 more
[2021-04-22 15:41:00,235][XNIO-1 task-3] INFO AccessLog - [AzSuccess] - CheckAlive - https://127.0.0.1:8443/check_alive - uid=Anonymous,o=Tremolo -
[127.0.0.1] - [f6d91ec095701342477177e10ca05138c102b439f]
[2021-04-22 15:41:00,375][XNIO-1 task-3] INFO AccessLog - [AzSuccess] - k8sIdp - https://127.0.0.1:8443/auth/idp/k8sIdp/.well-known/openid-configuration - uid=Anonymous,o=Tremolo - NONE [127.0.0.1] - [f1624a01a4d2c91023e68144bcc8ccd059b35f7a1]
sorry for the delay.
[2021-04-22 15:40:58,093][Thread-9] ERROR K8sWatcher - Could not get authentication token
javax.net.ssl.SSLException: Connection reset
You can ignore this, we're going to make this less verbose. This is because AKS has a really short timeout but we recover from it. Looks much worse then it is
Dashboard works till login. Namespaces not displaying.
In the dashboard do you see an error in the upper right hand corner? Chances are it's an RBAC issue