Websocket authentication errors

[Rishabh Bector]

I set up a mattermost chat server, which has its own helm chart: https://github.com/mattermost/mattermost-kubernetes

This helm chart includes an Nginx ingress controller, however I wanted to use the cloudflare ingress controller w/ argo tunnels. So, after installing the mattermost helm release, I deleted the default ingress controller and installed our one with this config (note that the service name changes every time):

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: argo-tunnel
    argo.cloudflare.com/lb-pool: testpool-name
  name: chat
  namespace: default
spec:
  rules:
  - host: chat.rishabh.ga
    http:
      paths:
      - backend:
          serviceName: wandering-frog-mattermost-app
          servicePort: 8065

This unfortunately doesn't work. I'm able to access the site at chat.rishabh.ga, but I see these logs in the chrome network tab:

websocket re-established connection
websocket_client.jsx:65 websocket closed
websocket_client.jsx:49 websocket re-established connection
websocket_client.jsx:65 websocket closed
websocket_client.jsx:49 websocket re-established connection
websocket_client.jsx:65 websocket closed

These logs in the cloudflare ingress controller pod:

time="2018-07-06T16:30:38Z" level=info msg="200 OK"
time="2018-07-06T16:30:38Z" level=warning msg="All requests should have a CF-RAY header. Please open a support ticket with Cloudflare. GET http://wandering-frog-mattermost-app.default:8065/ HTTP/1.1 "
time="2018-07-06T16:30:38Z" level=info msg="200 OK"
time="2018-07-06T16:30:39Z" level=warning msg="All requests should have a CF-RAY header. Please open a support ticket with Cloudflare. GET http://wandering-frog-mattermost-app.default:8065/ HTTP/1.1 "
time="2018-07-06T16:30:39Z" level=info msg="200 OK"
time="2018-07-06T16:30:39Z" level=warning msg="All requests should have a CF-RAY header. Please open a support ticket with Cloudflare. GET http://wandering-frog-mattermost-app.default:8065/ HTTP/1.1 "
time="2018-07-06T16:30:39Z" level=info msg="200 OK"
time="2018-07-06T16:30:41Z" level=warning msg="All requests should have a CF-RAY header. Please open a support ticket with Cloudflare. GET http://wandering-frog-mattermost-app.default:8065/ HTTP/1.1 "

And these in the mattermost app pod:

{"level":"error","ts":1530893950.0812402,"caller":"app/websocket_router.go:94","msg":"websocket routing error: seq=1 uid= api.web_socket_router.not_authenticated.app_error [details: ]"}

I'm not sure what's causing this. Maybe looking at the default ingress controller defined in the mattermost chart repo would help?

[Rishabh Bector]

Hi, is there any update on this?

[Nate Franzen]

I can reproduce the scenario with the mattermost helm chart.

• most of the functions of the the chat app seem to work
• some of the details (adding teams) do not show up when accessed through the argo tunnel/ingress controller
• the browser developer console shows the websocket being closed and reconnected
• the app is completely functional when accessed with port-forward to a pod, or via a nodeport service (in contrast to going through the cloudflare argo endpoint)
• the behavior is the same setting kube-proxy to iptables or ipvs, and setting scheduler to source hash, and only one pod replica is used.

I think the problem will will be easier to isolate if we use a simpler websocket application, without all of the mattermost complexity.