Crashloop with no logs between querying provider and verifying client
xunholy opened this issue · comments
There is an issue where the pods run into a crashloop because the start-up is never successful past the querying provider where it should continue moving into verifying client.
2021/11/04 05:45:35 Using config file: /app/config.yaml
2021/11/04 05:45:35 Creating new provider https://dex.example.com
This never progresses, I can confirm the pod can reach the provider and that it has connectivity, I've enabled debug: true yet this is kind of useless and doesn't add any extra verbosity to this phase of the pod life cycle.
Only clue i could consider is it's failing silently somewhere around here
Line 204 in 7b6588b
Open to suggestions on how or why this could be the case. My helm values can be found here https://github.com/xUnholy/k8s-gitops/blob/ef4cfeb9d9992e05b5b861fd2cb706511e8e7bf1/k8s/namespaces/base/network-system/dex-k8s-authenticator/helmrelease.yaml#L33
Is this related to #182 ? Since in that issue it looks like it got further than this ...and then errored out.
For this particular case, does it hang or does it exit? If it hangs then it sounds like it's stuck trying to get a connection (I've seen similar issue that have been the result of networkpolicy...).
You might have some luck with GODEBUG=http2debug=2 (this is part of net/http I think, which is what oidc.NewProvider uses).
There's more:
GODEBUG=http2client=0 # disable HTTP/2 client support
GODEBUG=http2server=0 # disable HTTP/2 server support
GODEBUG=http2debug=1 # enable verbose HTTP/2 debug logs
GODEBUG=http2debug=2 # ... even more verbose, with frame dumps