Docker 24.0.5 - Traefik doesn't accept any connections / server stuck
hhessel opened this issue · comments
Hi all
I just had to forcefully downgrade to version Docker version 24.0.4. and it worked again. Unfortunately it doesn't show any errors in the log-file. Any connection on 24.0.5 to Traffic opens but you can't send anything over the connection and it just time outs after a minute. I don't know if there are any breaking changes in 24.0.4 but it looks something broke.
Thanks
{"level":"debug","msg":"Provider event received {Status:health_status: healthy ID:d3e5cb7be921f39dd5e4d99d2e03d9990b70160bd5cc06b79b4a27c62063922e From:pihole/pihole:latest Type:container Action:health_status: healthy Actor:{ID:d3e5cb7be921f39dd5e4d99d2e03d9990b70160bd5cc06b79b4a27c62063922e Attributes:map[com.docker.compose.config-hash:e19dcbdd0bd212ebde1f0fdfc8e6116c4f06afb239be17c726a3fccda4b7e7bf com.docker.compose.container-number:1 com.docker.compose.depends_on: com.docker.compose.image:sha256:566a5a3d47738903cc1505a997c42cd98f059ccddd8ebf10ad505972fa568b88 com.docker.compose.oneoff:False com.docker.compose.project:pihole com.docker.compose.project.config_files:/srv/samba/docker/compose/pihole/docker-compose.yml com.docker.compose.project.working_dir:/srv/samba/docker/compose/pihole com.docker.compose.service:pihole com.docker.compose.version:2.18.1 image:pihole/pihole:latest name:pihole org.opencontainers.image.created:2023-05-30T18:51:05.660Z org.opencontainers.image.description:Pi-hole in a docker container org.opencontainers.image.licenses: org.opencontainers.image.revision:37bd3d2f0f9321211d9370fc96b7d06325cee887 org.opencontainers.image.source:https://github.com/pi-hole/docker-pi-hole org.opencontainers.image.title:docker-pi-hole org.opencontainers.image.url:https://github.com/pi-hole/docker-pi-hole org.opencontainers.image.version:2023.05.2 sh.acme.autoload.domain:pihole.XXXXXX.COM traefik.docker.network:custom_bridge traefik.enable:true traefik.http.routers.pihole-router.entrypoints:websecure traefik.http.routers.pihole-router.rule:Host(`pihole.XXXXXX.COM`) traefik.http.routers.pihole-router.service:pihole-service traefik.http.routers.pihole-router.tls:true traefik.http.services.pihole-service.loadbalancer.server.port:8081]} Scope:local Time:1690268863 TimeNano:1690268863367647508}","providerName":"docker","time":"2023-07-25T09:07:43+02:00"}
{"container":"traefik-traefik-a479241e79b8f36a8e663485c706a451a7685a8ea97c48de49b26f25e76c1d56","level":"debug","msg":"Filtering disabled container","providerName":"docker","time":"2023-07-25T09:07:43+02:00"}
{"container":"tika-lab-83676627b38892653ff27b8c8f52e3be27c68bf64ee05c030ea7a1c8435d9ed4","level":"debug","msg":"Filtering disabled container","providerName":"docker","time":"2023-07-25T09:07:43+02:00"}
{"container":"plex-plex-b48894c4c29e5b1ed560c657d5e37cd4ca4aafb79e7433494fdc64cddcf010de","level":"debug","msg":"Filtering disabled container","providerName":"docker","time":"2023-07-25T09:07:43+02:00"}
{"level":"debug","msg":"Configuration received: {\"http\":{\"routers\":{\"lab-router\":{\"service\":\"lab-service\",\"rule\":\"Host(`lab.XXXXXX.COM`)\",\"tls\":{}},\"labapi-router\":{\"service\":\"labapi-service\",\"rule\":\"Host(`api.XXXXXX.COM`)\",\"tls\":{}},\"pihole-router\":{\"entryPoints\":[\"websecure\"],\"service\":\"pihole-service\",\"rule\":\"Host(`pihole.XXXXXX.COM`)\",\"tls\":{}},\"portainer-router\":{\"entryPoints\":[\"websecure\"],\"service\":\"portainer-service\",\"rule\":\"Host(`portainer.XXXXXX.COM`)\",\"tls\":{}}},\"services\":{\"lab-service\":{\"loadBalancer\":{\"servers\":[{\"url\":\"http://172.20.0.5:8888\"}],\"passHostHeader\":true}},\"labapi-service\":{\"loadBalancer\":{\"servers\":[{\"url\":\"http://172.20.0.5:5000\"}],\"passHostHeader\":true}},\"pihole-service\":{\"loadBalancer\":{\"servers\":[{\"url\":\"http://172.20.0.6:8081\"}],\"passHostHeader\":true}},\"portainer-service\":{\"loadBalancer\":{\"servers\":[{\"url\":\"http://172.20.0.4:9000\"}],\"passHostHeader\":true}}}},\"tcp\":{},\"udp\":{}}","providerName":"docker","time":"2023-07-25T09:07:43+02:00"}
{"level":"debug","msg":"No entryPoint defined for this router, using the default one(s) instead: [web websecure]","routerName":"lab-router","time":"2023-07-25T09:07:43+02:00"}
{"level":"debug","msg":"No entryPoint defined for this router, using the default one(s) instead: [web websecure]","routerName":"labapi-router","time":"2023-07-25T09:07:43+02:00"}
{"level":"debug","msg":"No store is defined to add the certificate MIIE7TCCA9WgAwIBAgISA5fHLHvI0ancHbY44rDk3XMOMA0GCS, it will be added to the default store.","time":"2023-07-25T09:07:43+02:00"}
{"level":"debug","msg":"Adding certificate for domain(s) *.XXXXXX.COM","time":"2023-07-25T09:07:43+02:00"}
{"level":"debug","msg":"No default certificate, fallback to the internal generated certificate","time":"2023-07-25T09:07:43+02:00","tlsStoreName":"default"}
{"entryPointName":"web","level":"debug","middlewareName":"tracing","middlewareType":"TracingForwarder","msg":"Added outgoing tracing middleware noop@internal","routerName":"web-to-websecure@internal","time":"2023-07-25T09:07:43+02:00"}
{"entryPointName":"web","level":"debug","middlewareName":"redirect-web-to-websecure@internal","middlewareType":"RedirectScheme","msg":"Creating middleware","routerName":"web-to-websecure@internal","time":"2023-07-25T09:07:43+02:00"}
{"entryPointName":"web","level":"debug","middlewareName":"redirect-web-to-websecure@internal","middlewareType":"RedirectScheme","msg":"Setting up redirection to https 443","routerName":"web-to-websecure@internal","time":"2023-07-25T09:07:43+02:00"}
{"entryPointName":"web","level":"debug","middlewareName":"traefik-internal-recovery","middlewareType":"Recovery","msg":"Creating middleware","time":"2023-07-25T09:07:43+02:00"}
{"entryPointName":"websecure","level":"debug","middlewareName":"pipelining","middlewareType":"Pipelining","msg":"Creating middleware","routerName":"ds@file","serviceName":"service-ds","time":"2023-07-25T09:07:43+02:00"}
{"entryPointName":"websecure","level":"debug","msg":"Creating load-balancer","routerName":"ds@file","serviceName":"service-ds","time":"2023-07-25T09:07:43+02:00"}
{"entryPointName":"websecure","level":"debug","msg":"Creating server 0 http://192.168.2.198:5500","routerName":"ds@file","serverName":0,"serviceName":"service-ds","time":"2023-07-25T09:07:43+02:00"}
{"level":"debug","msg":"child http://192.168.2.198:5500 now UP","time":"2023-07-25T09:07:43+02:00"}
{"level":"debug","msg":"Propagating new UP status","time":"2023-07-25T09:07:43+02:00"}
{"entryPointName":"websecure","level":"debug","middlewareName":"tracing","middlewareType":"TracingForwarder","msg":"Added outgoing tracing middleware service-ds","routerName":"ds@file","time":"2023-07-25T09:07:43+02:00"}
{"entryPointName":"websecure","level":"debug","middlewareName":"tracing","middlewareType":"TracingForwarder","msg":"Added outgoing tracing middleware api@internal","routerName":"api@file","time":"2023-07-25T09:07:43+02:00"}
{"entryPointName":"websecure","level":"debug","middlewareName":"auth@file","middlewareType":"BasicAuth","msg":"Creating middleware","routerName":"api@file","time":"2023-07-25T09:07:43+02:00"}
{"entryPointName":"websecure","level":"debug","middlewareName":"auth@file","msg":"Adding tracing to middleware","routerName":"api@file","time":"2023-07-25T09:07:43+02:00"}
{"entryPointName":"websecure","level":"debug","middlewareName":"pipelining","middlewareType":"Pipelining","msg":"Creating middleware","routerName":"pihole-router@docker","serviceName":"pihole-service","time":"2023-07-25T09:07:43+02:00"}
{"entryPointName":"websecure","level":"debug","msg":"Creating load-balancer","routerName":"pihole-router@docker","serviceName":"pihole-service","time":"2023-07-25T09:07:43+02:00"}
{"entryPointName":"websecure","level":"debug","msg":"Creating server 0 http://172.20.0.6:8081","routerName":"pihole-router@docker","serverName":0,"serviceName":"pihole-service","time":"2023-07-25T09:07:43+02:00"}
{"level":"debug","msg":"child http://172.20.0.6:8081 now UP","time":"2023-07-25T09:07:43+02:00"}
{"level":"debug","msg":"Propagating new UP status","time":"2023-07-25T09:07:43+02:00"}
{"entryPointName":"websecure","level":"debug","middlewareName":"tracing","middlewareType":"TracingForwarder","msg":"Added outgoing tracing middleware pihole-service","routerName":"pihole-router@docker","time":"2023-07-25T09:07:43+02:00"}
{"entryPointName":"websecure","level":"debug","middlewareName":"pipelining","middlewareType":"Pipelining","msg":"Creating middleware","routerName":"portainer-router@docker","serviceName":"portainer-service","time":"2023-07-25T09:07:43+02:00"}
{"entryPointName":"websecure","level":"debug","msg":"Creating load-balancer","routerName":"portainer-router@docker","serviceName":"portainer-service","time":"2023-07-25T09:07:43+02:00"}
{"entryPointName":"websecure","level":"debug","msg":"Creating server 0 http://172.20.0.4:9000","routerName":"portainer-router@docker","serverName":0,"serviceName":"portainer-service","time":"2023-07-25T09:07:43+02:00"}
{"level":"debug","msg":"child http://172.20.0.4:9000 now UP","time":"2023-07-25T09:07:43+02:00"}
{"level":"debug","msg":"Propagating new UP status","time":"2023-07-25T09:07:43+02:00"}
{"entryPointName":"websecure","level":"debug","middlewareName":"tracing","middlewareType":"TracingForwarder","msg":"Added outgoing tracing middleware portainer-service","routerName":"portainer-router@docker","time":"2023-07-25T09:07:43+02:00"}
{"entryPointName":"websecure","level":"debug","middlewareName":"pipelining","middlewareType":"Pipelining","msg":"Creating middleware","routerName":"lab-router@docker","serviceName":"lab-service","time":"2023-07-25T09:07:43+02:00"}
{"entryPointName":"websecure","level":"debug","msg":"Creating load-balancer","routerName":"lab-router@docker","serviceName":"lab-service","time":"2023-07-25T09:07:43+02:00"}
{"entryPointName":"websecure","level":"debug","msg":"Creating server 0 http://172.20.0.5:8888","routerName":"lab-router@docker","serverName":0,"serviceName":"lab-service","time":"2023-07-25T09:07:43+02:00"}
{"level":"debug","msg":"child http://172.20.0.5:8888 now UP","time":"2023-07-25T09:07:43+02:00"}
{"level":"debug","msg":"Propagating new UP status","time":"2023-07-25T09:07:43+02:00"}
{"entryPointName":"websecure","level":"debug","middlewareName":"tracing","middlewareType":"TracingForwarder","msg":"Added outgoing tracing middleware lab-service","routerName":"lab-router@docker","time":"2023-07-25T09:07:43+02:00"}
{"entryPointName":"websecure","level":"debug","middlewareName":"pipelining","middlewareType":"Pipelining","msg":"Creating middleware","routerName":"labapi-router@docker","serviceName":"labapi-service","time":"2023-07-25T09:07:43+02:00"}
{"entryPointName":"websecure","level":"debug","msg":"Creating load-balancer","routerName":"labapi-router@docker","serviceName":"labapi-service","time":"2023-07-25T09:07:43+02:00"}
{"entryPointName":"websecure","level":"debug","msg":"Creating server 0 http://172.20.0.5:5000","routerName":"labapi-router@docker","serverName":0,"serviceName":"labapi-service","time":"2023-07-25T09:07:43+02:00"}
{"level":"debug","msg":"child http://172.20.0.5:5000 now UP","time":"2023-07-25T09:07:43+02:00"}
{"level":"debug","msg":"Propagating new UP status","time":"2023-07-25T09:07:43+02:00"}
{"entryPointName":"websecure","level":"debug","middlewareName":"tracing","middlewareType":"TracingForwarder","msg":"Added outgoing tracing middleware labapi-service","routerName":"labapi-router@docker","time":"2023-07-25T09:07:43+02:00"}
{"entryPointName":"websecure","level":"debug","middlewareName":"pipelining","middlewareType":"Pipelining","msg":"Creating middleware","routerName":"asus@file","serviceName":"service-asus","time":"2023-07-25T09:07:43+02:00"}
{"entryPointName":"websecure","level":"debug","msg":"Creating load-balancer","routerName":"asus@file","serviceName":"service-asus","time":"2023-07-25T09:07:43+02:00"}
{"entryPointName":"websecure","level":"debug","msg":"Creating server 0 http://192.168.2.200:80","routerName":"asus@file","serverName":0,"serviceName":"service-asus","time":"2023-07-25T09:07:43+02:00"}
{"level":"debug","msg":"child http://192.168.2.200:80 now UP","time":"2023-07-25T09:07:43+02:00"}
{"level":"debug","msg":"Propagating new UP status","time":"2023-07-25T09:07:43+02:00"}
{"entryPointName":"websecure","level":"debug","middlewareName":"tracing","middlewareType":"TracingForwarder","msg":"Added outgoing tracing middleware service-asus","routerName":"asus@file","time":"2023-07-25T09:07:43+02:00"}
{"entryPointName":"websecure","level":"debug","middlewareName":"traefik-internal-recovery","middlewareType":"Recovery","msg":"Creating middleware","time":"2023-07-25T09:07:43+02:00"}
{"entryPointName":"web","level":"debug","middlewareName":"traefik-internal-recovery","middlewareType":"Recovery","msg":"Creating middleware","time":"2023-07-25T09:07:43+02:00"}
{"entryPointName":"web","level":"debug","msg":"Adding route for lab.XXXXXX.COM with TLS options default","time":"2023-07-25T09:07:43+02:00"}
{"entryPointName":"web","level":"debug","msg":"Adding route for api.XXXXXX.COM with TLS options default","time":"2023-07-25T09:07:43+02:00"}
{"entryPointName":"websecure","level":"debug","msg":"Adding route for portainer.XXXXXX.COM with TLS options default","time":"2023-07-25T09:07:43+02:00"}
{"entryPointName":"websecure","level":"debug","msg":"Adding route for lab.XXXXXX.COM with TLS options default","time":"2023-07-25T09:07:43+02:00"}
{"entryPointName":"websecure","level":"debug","msg":"Adding route for api.XXXXXX.COM with TLS options default","time":"2023-07-25T09:07:43+02:00"}
{"entryPointName":"websecure","level":"debug","msg":"Adding route for asus.XXXXXX.COM with TLS options default","time":"2023-07-25T09:07:43+02:00"}
{"entryPointName":"websecure","level":"debug","msg":"Adding route for ds-web.XXXXXX.COM with TLS options default","time":"2023-07-25T09:07:43+02:00"}
{"entryPointName":"websecure","level":"debug","msg":"Adding route for monitor.XXXXXX.COM with TLS options default","time":"2023-07-25T09:07:43+02:00"}
{"entryPointName":"websecure","level":"debug","msg":"Adding route for pihole.XXXXXX.COM with TLS options default","time":"2023-07-25T09:07:43+02:00"}
Hello,
I think your problem is related to:
The Go client now avoids using UNIX socket paths in the HTTP Host: header, in order to be compatible with changes introduced in go1.20.6.
https://github.com/moby/moby/releases/tag/v24.0.5
Can you provide your configuration?
Hi, attached my compose file. It's pretty straightforward.
version: '3.1'
networks:
custom_bridge:
external: true
services:
traefik:
image: traefik
container_name: tr
networks:
custom_bridge:
volumes:
- /srv/samba/docker/appdata/traefik:/etc/traefik
- /var/run/docker.sock:/var/run/docker.sock:ro
ports:
- 80:80
- 443:443
- 8080:8080
environment:
- TZ=Europe/Zurich
labels:
"sh.acme.autoload.domain": "traefik.XXX"
restart: unless-stopped
Can you provide your static configuration?
Sure,
rules.yaml
http:
routers:
api:
rule: Host(`monitor.xxxx.com`) && (PathPrefix(`/api`) || PathPrefix(`/dashboard`))
service: api@internal
tls: {}
entryPoints:
- "websecure"
middlewares:
- auth
ds:
rule: Host(`ds-web.xxxx.com`)
service: service-ds
tls: {}
entryPoints:
- "websecure"
asus:
rule: Host(`asus.xxxx.com`)
service: service-asus
tls: {}
entryPoints:
- "websecure"
middlewares:
auth:
basicAuth:
users:
- "hhessel:XXXX"
services:
service-ds:
loadBalancer:
servers:
- url: "http://192.168.2.198:5500"
service-asus:
loadBalancer:
servers:
- url: "http://192.168.2.200:80"
traefik.yaml
entryPoints:
web:
address: :80
http:
redirections:
entryPoint:
to: websecure
scheme: https
permanent: true
websecure:
address: :443
traefik:
address: :8080
providers:
file:
directory: /etc/traefik
watch: true
docker:
endpoint: unix:///var/run/docker.sock
watch: true
exposedByDefault: false
api:
dashboard: true
log:
level: DEBUG
filePath: "/etc/traefik/logs/log-file.log"
format: json
can you open an issue on the Traefik repository and fill in the issue template with your information?
https://github.com/traefik/traefik/issues/new?assignees=&labels=&projects=&template=bug_report.yml
Thanks for the first triage @ldez . Issue raised.