Websocket authentication errors
rishabh-bector opened this issue · comments
I set up a mattermost chat server, which has its own helm chart: https://github.com/mattermost/mattermost-kubernetes
This helm chart includes an Nginx ingress controller, however I wanted to use the cloudflare ingress controller w/ argo tunnels. So, after installing the mattermost helm release, I deleted the default ingress controller and installed our one with this config (note that the service name changes every time):
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: argo-tunnel
argo.cloudflare.com/lb-pool: testpool-name
name: chat
namespace: default
spec:
rules:
- host: chat.rishabh.ga
http:
paths:
- backend:
serviceName: wandering-frog-mattermost-app
servicePort: 8065
This unfortunately doesn't work. I'm able to access the site at chat.rishabh.ga, but I see these logs in the chrome network tab:
websocket re-established connection
websocket_client.jsx:65 websocket closed
websocket_client.jsx:49 websocket re-established connection
websocket_client.jsx:65 websocket closed
websocket_client.jsx:49 websocket re-established connection
websocket_client.jsx:65 websocket closed
These logs in the cloudflare ingress controller pod:
time="2018-07-06T16:30:38Z" level=info msg="200 OK"
time="2018-07-06T16:30:38Z" level=warning msg="All requests should have a CF-RAY header. Please open a support ticket with Cloudflare. GET http://wandering-frog-mattermost-app.default:8065/ HTTP/1.1 "
time="2018-07-06T16:30:38Z" level=info msg="200 OK"
time="2018-07-06T16:30:39Z" level=warning msg="All requests should have a CF-RAY header. Please open a support ticket with Cloudflare. GET http://wandering-frog-mattermost-app.default:8065/ HTTP/1.1 "
time="2018-07-06T16:30:39Z" level=info msg="200 OK"
time="2018-07-06T16:30:39Z" level=warning msg="All requests should have a CF-RAY header. Please open a support ticket with Cloudflare. GET http://wandering-frog-mattermost-app.default:8065/ HTTP/1.1 "
time="2018-07-06T16:30:39Z" level=info msg="200 OK"
time="2018-07-06T16:30:41Z" level=warning msg="All requests should have a CF-RAY header. Please open a support ticket with Cloudflare. GET http://wandering-frog-mattermost-app.default:8065/ HTTP/1.1 "
And these in the mattermost app pod:
{"level":"error","ts":1530893950.0812402,"caller":"app/websocket_router.go:94","msg":"websocket routing error: seq=1 uid= api.web_socket_router.not_authenticated.app_error [details: ]"}
I'm not sure what's causing this. Maybe looking at the default ingress controller defined in the mattermost chart repo would help?
Hi, is there any update on this?
I can reproduce the scenario with the mattermost helm chart.
- most of the functions of the the chat app seem to work
- some of the details (adding teams) do not show up when accessed through the argo tunnel/ingress controller
- the browser developer console shows the websocket being closed and reconnected
- the app is completely functional when accessed with port-forward to a pod, or via a nodeport service (in contrast to going through the cloudflare argo endpoint)
- the behavior is the same setting kube-proxy to iptables or ipvs, and setting scheduler to source hash, and only one pod replica is used.
I think the problem will will be easier to isolate if we use a simpler websocket application, without all of the mattermost complexity.