NAT stop working for incomming connections after restart
VenusGen opened this issue · comments
I wrote Caddy and ipv6nat in a docker-compose file, It works very well. But after I executed the docker-compose restart command, I am still able to ping other v6 addresses from the Caddy container but can't receive any more incoming requests (It was able to receive when userland-proxy enabled, but Caddy couldn't get the real IP address).
I think it's probably similar to #14 , probably we need to remove iptables when container stopping?
ip6tables -L:
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain FORWARD (policy ACCEPT)
target prot opt source destination
DOCKER-USER all anywhere anywhere
DOCKER-ISOLATION-STAGE-1 all anywhere anywhere
DOCKER all anywhere anywhere
ACCEPT all anywhere anywhere ctstate RELATED,ESTABLISHED
ACCEPT all anywhere anywhere
ACCEPT all anywhere anywhere
DOCKER all anywhere anywhere
ACCEPT all anywhere anywhere ctstate RELATED,ESTABLISHED
ACCEPT all anywhere anywhere
ACCEPT all anywhere anywhere
DOCKER all anywhere anywhere
ACCEPT all anywhere anywhere ctstate RELATED,ESTABLISHED
ACCEPT all anywhere anywhere
ACCEPT all anywhere anywhere
DOCKER all anywhere anywhere
ACCEPT all anywhere anywhere ctstate RELATED,ESTABLISHED
ACCEPT all anywhere anywhere
ACCEPT all anywhere anywhere
DOCKER all anywhere anywhere
ACCEPT all anywhere anywhere ctstate RELATED,ESTABLISHED
ACCEPT all anywhere anywhere
ACCEPT all anywhere anywhere
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
Chain DOCKER (5 references)
target prot opt source destination
ACCEPT tcp anywhere fd00:beef::3 tcp dpt:https
ACCEPT tcp anywhere fd00:beef::3 tcp dpt:http
Chain DOCKER-ISOLATION-STAGE-1 (1 references)
target prot opt source destination
DOCKER-ISOLATION-STAGE-2 all anywhere anywhere
RETURN all anywhere anywhere
Chain DOCKER-ISOLATION-STAGE-2 (1 references)
target prot opt source destination
DROP all anywhere anywhere
RETURN all anywhere anywhere
Chain DOCKER-USER (1 references)
target prot opt source destination
RETURN all anywhere anywhere
docker-compose.yml:
version: '2.1'
services:
caddy:
image: abiosoft/caddy
container_name: caddy
ports:
- "80:80"
- "443:443"
environment:
- CADDYPATH=/caddy
- ACME_AGREE=true
volumes:
- ./Caddyfile:/etc/Caddyfile
- caddyacme:/caddy/acme
restart: always
networks:
v6net:
depends_on:
- ipv6nat
ipv6nat:
image: robbertkl/ipv6nat
restart: always
network_mode: "host"
cap_add:
- NET_ADMIN
- SYS_MODULE
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
networks:
v6net:
driver: bridge
enable_ipv6: true
ipam:
driver: default
config:
- subnet: 172.20.0.0/16
gateway: 172.20.0.1
- subnet: fd00:beef::/80
volumes:
caddyacme:
Docker Environment:
Client: Docker Engine - Community
Version: 19.03.2
API version: 1.40
Go version: go1.12.8
Git commit: 6a30dfc
Built: Thu Aug 29 05:29:11 2019
OS/Arch: linux/amd64
Experimental: false
Server: Docker Engine - Community
Engine:
Version: 19.03.2
API version: 1.40 (minimum version 1.12)
Go version: go1.12.8
Git commit: 6a30dfc
Built: Thu Aug 29 05:27:45 2019
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.2.6
GitCommit: 894b81a4b802e4eb2a91d1ce216b8817763c29fb
runc:
Version: 1.0.0-rc8
GitCommit: 425e105d5a03fabd737a126ad93d62a9eeede87f
docker-init:
Version: 0.18.0
GitCommit: fec3683
Docker Compose:
docker-compose version 1.24.1, build 4667896b
docker-py version: 3.7.3
CPython version: 3.6.8
OpenSSL version: OpenSSL 1.1.0j 20 Nov 2018
@VenusGen You should use the -cleanup flag if you start docker-ipv6nat. Otherwise there might be duplicated or intefering rules if you restart docker-ipv6nat. Can you test this?
It seems avoided duplicated rules now, but when it clear up the rules, it also wiped the 443 & 80 mapping...
➜ ip6tables -L
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain FORWARD (policy ACCEPT)
target prot opt source destination
DOCKER-USER all anywhere anywhere
DOCKER-ISOLATION-STAGE-1 all anywhere anywhere
DOCKER all anywhere anywhere
ACCEPT all anywhere anywhere ctstate RELATED,ESTABLISHED
ACCEPT all anywhere anywhere
ACCEPT all anywhere anywhere
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
**Chain DOCKER (1 references)
target prot opt source destination
ACCEPT tcp anywhere fd00:beef::3 tcp dpt:https
ACCEPT tcp anywhere fd00:beef::3 tcp dpt:http**
Chain DOCKER-ISOLATION-STAGE-1 (1 references)
target prot opt source destination
DOCKER-ISOLATION-STAGE-2 all anywhere anywhere
RETURN all anywhere anywhere
Chain DOCKER-ISOLATION-STAGE-2 (1 references)
target prot opt source destination
DROP all anywhere anywhere
RETURN all anywhere anywhere
Chain DOCKER-USER (1 references)
target prot opt source destination
RETURN all anywhere anywhere
➜ docker-compose restart
Restarting caddy ... done
Restarting ipv6nat ... done
➜ ip6tables -L
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain FORWARD (policy ACCEPT)
target prot opt source destination
DOCKER-USER all anywhere anywhere
DOCKER-ISOLATION-STAGE-1 all anywhere anywhere
DOCKER all anywhere anywhere
ACCEPT all anywhere anywhere ctstate RELATED,ESTABLISHED
ACCEPT all anywhere anywhere
ACCEPT all anywhere anywhere
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
Chain DOCKER (1 references)
target prot opt source destination
Chain DOCKER-ISOLATION-STAGE-1 (1 references)
target prot opt source destination
DOCKER-ISOLATION-STAGE-2 all anywhere anywhere
RETURN all anywhere anywhere
Chain DOCKER-ISOLATION-STAGE-2 (1 references)
target prot opt source destination
DROP all anywhere anywhere
RETURN all anywhere anywhere
Chain DOCKER-USER (1 references)
target prot opt source destination
RETURN all anywhere anywhere
Could you please use the output of ip6tables-save instead of ip6tables -L? This would give us the full rules.
Not sure why the 80/443 rules are missing. Is your docker-compose.yml still like in your first post? Could you run ipv6nat with the -debug flag and take a look at the output?
Is your docker-compose.yml still like in your first post?
No, I add a line command: --retry -cleanup -debug after the volumes part of ipv6nat to run with those flags.
The full logs are attached below. Briefly, the log shows v6nat removed 443&80 but never add them back.
➜ ip6tables-save
# Generated by ip6tables-save v1.6.1 on Mon Nov 18 09:39:57 2019
*nat
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
:DOCKER - [0:0]
-A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER
-A OUTPUT -m addrtype --dst-type LOCAL -j DOCKER
-A OUTPUT ! -d ::1/128 -m addrtype --dst-type LOCAL -j DOCKER
-A POSTROUTING -o br-0d708b1e1de8 -m addrtype --dst-type LOCAL -j MASQUERADE
-A POSTROUTING -s fd00:beef::/80 ! -o br-0d708b1e1de8 -j MASQUERADE
-A POSTROUTING -s fd00:beef::3/128 -d fd00:beef::3/128 -p tcp -m tcp --dport 443 -j MASQUERADE
-A POSTROUTING -s fd00:beef::3/128 -d fd00:beef::3/128 -p tcp -m tcp --dport 80 -j MASQUERADE
-A POSTROUTING -o br-1aa0f87300b3 -m addrtype --dst-type LOCAL -j MASQUERADE
-A POSTROUTING -s fd00:beef::/80 ! -o br-1aa0f87300b3 -j MASQUERADE
-A POSTROUTING -o br-50ddbadc04b4 -m addrtype --dst-type LOCAL -j MASQUERADE
-A POSTROUTING -s fd00:beef::/80 ! -o br-50ddbadc04b4 -j MASQUERADE
-A POSTROUTING -o br-c11ddedb2910 -m addrtype --dst-type LOCAL -j MASQUERADE
-A POSTROUTING -s fd00:beef::/80 ! -o br-c11ddedb2910 -j MASQUERADE
-A POSTROUTING -o br-e59b5f972548 -m addrtype --dst-type LOCAL -j MASQUERADE
-A POSTROUTING -s fd00:beef::/80 ! -o br-e59b5f972548 -j MASQUERADE
-A POSTROUTING -o br-37f1ec58650a -m addrtype --dst-type LOCAL -j MASQUERADE
-A POSTROUTING -s fd00:beef::/80 ! -o br-37f1ec58650a -j MASQUERADE
-A POSTROUTING -s fd00:beef::2/128 -d fd00:beef::2/128 -p tcp -m tcp --dport 443 -j MASQUERADE
-A POSTROUTING -s fd00:beef::2/128 -d fd00:beef::2/128 -p tcp -m tcp --dport 80 -j MASQUERADE
-A POSTROUTING -o br-1b81271987dc -m addrtype --dst-type LOCAL -j MASQUERADE
-A POSTROUTING -s fd00:beef::/80 ! -o br-1b81271987dc -j MASQUERADE
-A DOCKER -p tcp -m tcp --dport 443 -j DNAT --to-destination [fd00:beef::3]:443
-A DOCKER -p tcp -m tcp --dport 80 -j DNAT --to-destination [fd00:beef::3]:80
COMMIT
# Completed on Mon Nov 18 09:39:57 2019
# Generated by ip6tables-save v1.6.1 on Mon Nov 18 09:39:57 2019
*filter
:INPUT ACCEPT [10:576]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [6:520]
:DOCKER - [0:0]
:DOCKER-ISOLATION-STAGE-1 - [0:0]
:DOCKER-ISOLATION-STAGE-2 - [0:0]
:DOCKER-USER - [0:0]
-A FORWARD -j DOCKER-USER
-A FORWARD -j DOCKER-ISOLATION-STAGE-1
-A FORWARD -o br-0d708b1e1de8 -j DOCKER
-A FORWARD -o br-0d708b1e1de8 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -i br-0d708b1e1de8 ! -o br-0d708b1e1de8 -j ACCEPT
-A FORWARD -i br-0d708b1e1de8 -o br-0d708b1e1de8 -j ACCEPT
-A DOCKER -d fd00:beef::3/128 ! -i br-0d708b1e1de8 -o br-0d708b1e1de8 -p tcp -m tcp --dport 443 -j ACCEPT
-A DOCKER -d fd00:beef::3/128 ! -i br-0d708b1e1de8 -o br-0d708b1e1de8 -p tcp -m tcp --dport 80 -j ACCEPT
-A DOCKER-ISOLATION-STAGE-1 -i br-0d708b1e1de8 ! -o br-0d708b1e1de8 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -j RETURN
-A DOCKER-ISOLATION-STAGE-2 -o br-0d708b1e1de8 -j DROP
-A DOCKER-ISOLATION-STAGE-2 -j RETURN
-A DOCKER-USER -j RETURN
COMMIT
# Completed on Mon Nov 18 09:39:57 2019
➜ docker-compose restart
Restarting caddy ... done
Restarting ipv6nat ... done
➜ ip6tables-save
# Generated by ip6tables-save v1.6.1 on Mon Nov 18 09:40:21 2019
*nat
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
:DOCKER - [0:0]
-A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER
-A OUTPUT -m addrtype --dst-type LOCAL -j DOCKER
-A OUTPUT ! -d ::1/128 -m addrtype --dst-type LOCAL -j DOCKER
-A POSTROUTING -o br-0d708b1e1de8 -m addrtype --dst-type LOCAL -j MASQUERADE
-A POSTROUTING -s fd00:beef::/80 ! -o br-0d708b1e1de8 -j MASQUERADE
-A POSTROUTING -o br-1aa0f87300b3 -m addrtype --dst-type LOCAL -j MASQUERADE
-A POSTROUTING -s fd00:beef::/80 ! -o br-1aa0f87300b3 -j MASQUERADE
-A POSTROUTING -o br-50ddbadc04b4 -m addrtype --dst-type LOCAL -j MASQUERADE
-A POSTROUTING -s fd00:beef::/80 ! -o br-50ddbadc04b4 -j MASQUERADE
-A POSTROUTING -o br-c11ddedb2910 -m addrtype --dst-type LOCAL -j MASQUERADE
-A POSTROUTING -s fd00:beef::/80 ! -o br-c11ddedb2910 -j MASQUERADE
-A POSTROUTING -o br-e59b5f972548 -m addrtype --dst-type LOCAL -j MASQUERADE
-A POSTROUTING -s fd00:beef::/80 ! -o br-e59b5f972548 -j MASQUERADE
-A POSTROUTING -o br-37f1ec58650a -m addrtype --dst-type LOCAL -j MASQUERADE
-A POSTROUTING -s fd00:beef::/80 ! -o br-37f1ec58650a -j MASQUERADE
-A POSTROUTING -s fd00:beef::2/128 -d fd00:beef::2/128 -p tcp -m tcp --dport 443 -j MASQUERADE
-A POSTROUTING -s fd00:beef::2/128 -d fd00:beef::2/128 -p tcp -m tcp --dport 80 -j MASQUERADE
-A POSTROUTING -o br-1b81271987dc -m addrtype --dst-type LOCAL -j MASQUERADE
-A POSTROUTING -s fd00:beef::/80 ! -o br-1b81271987dc -j MASQUERADE
COMMIT
# Completed on Mon Nov 18 09:40:21 2019
# Generated by ip6tables-save v1.6.1 on Mon Nov 18 09:40:21 2019
*filter
:INPUT ACCEPT [4:224]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [16:1568]
:DOCKER - [0:0]
:DOCKER-ISOLATION-STAGE-1 - [0:0]
:DOCKER-ISOLATION-STAGE-2 - [0:0]
:DOCKER-USER - [0:0]
-A FORWARD -j DOCKER-USER
-A FORWARD -j DOCKER-ISOLATION-STAGE-1
-A FORWARD -o br-0d708b1e1de8 -j DOCKER
-A FORWARD -o br-0d708b1e1de8 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -i br-0d708b1e1de8 ! -o br-0d708b1e1de8 -j ACCEPT
-A FORWARD -i br-0d708b1e1de8 -o br-0d708b1e1de8 -j ACCEPT
-A DOCKER-ISOLATION-STAGE-1 -i br-0d708b1e1de8 ! -o br-0d708b1e1de8 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -j RETURN
-A DOCKER-ISOLATION-STAGE-2 -o br-0d708b1e1de8 -j DROP
-A DOCKER-ISOLATION-STAGE-2 -j RETURN
-A DOCKER-USER -j RETURN
COMMIT
# Completed on Mon Nov 18 09:40:21 2019
➜ docker-compose logs
Attaching to caddy, ipv6nat
ipv6nat | # Warning: iptables-legacy tables present, use iptables-legacy to see them
ipv6nat | 2019/11/18 09:37:50 docker-ipv6nat is running in debug mode
ipv6nat | 2019/11/18 09:37:50 rule added: -t filter -A DOCKER-ISOLATION-STAGE-1 1 -j RETURN
ipv6nat | 2019/11/18 09:37:50 rule added: -t filter -A DOCKER-ISOLATION-STAGE-2 1 -j RETURN
ipv6nat | 2019/11/18 09:37:50 rule added: -t nat -A PREROUTING 1 -m addrtype --dst-type LOCAL -j DOCKER
ipv6nat | 2019/11/18 09:37:50 rule added: -t nat -A OUTPUT 1 -m addrtype --dst-type LOCAL -j DOCKER
ipv6nat | 2019/11/18 09:37:50 rule added: -t filter -A FORWARD 1 -j DOCKER-ISOLATION-STAGE-1
ipv6nat | 2019/11/18 09:37:50 rule added: -t filter -A FORWARD 1 -j DOCKER-USER
ipv6nat | 2019/11/18 09:37:50 rule added: -t filter -A FORWARD 3 -o br-0d708b1e1de8 -j DOCKER
ipv6nat | 2019/11/18 09:37:50 rule added: -t filter -A FORWARD 4 -o br-0d708b1e1de8 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
ipv6nat | 2019/11/18 09:37:50 rule added: -t filter -A FORWARD 5 -i br-0d708b1e1de8 ! -o br-0d708b1e1de8 -j ACCEPT
ipv6nat | 2019/11/18 09:37:50 rule added: -t filter -A FORWARD 6 -i br-0d708b1e1de8 -o br-0d708b1e1de8 -j ACCEPT
ipv6nat | 2019/11/18 09:37:50 rule added: -t nat -A POSTROUTING 1 -s fd00:beef::/80 ! -o br-0d708b1e1de8 -j MASQUERADE
ipv6nat | 2019/11/18 09:37:50 rule added: -t nat -A POSTROUTING 1 -o br-0d708b1e1de8 -m addrtype --dst-type LOCAL -j MASQUERADE
ipv6nat | 2019/11/18 09:37:50 rule added: -t filter -A DOCKER-ISOLATION-STAGE-2 1 -o br-0d708b1e1de8 -j DROP
ipv6nat | 2019/11/18 09:37:50 rule added: -t filter -A DOCKER-ISOLATION-STAGE-1 1 -i br-0d708b1e1de8 ! -o br-0d708b1e1de8 -j DOCKER-ISOLATION-STAGE-2
ipv6nat | 2019/11/18 09:37:53 rule added: -t filter -A DOCKER 1 -d fd00:beef::3 ! -i br-0d708b1e1de8 -o br-0d708b1e1de8 -p tcp -m tcp --dport 443 -j ACCEPT
ipv6nat | 2019/11/18 09:37:53 rule added: -t nat -A POSTROUTING 3 -s fd00:beef::3 -d fd00:beef::3 -p tcp -m tcp --dport 443 -j MASQUERADE
ipv6nat | 2019/11/18 09:37:53 rule added: -t nat -A DOCKER 1 -d 0/0 -p tcp -m tcp --dport 443 -j DNAT --to-destination [fd00:beef::3]:443
ipv6nat | 2019/11/18 09:37:53 rule added: -t filter -A DOCKER 2 -d fd00:beef::3 ! -i br-0d708b1e1de8 -o br-0d708b1e1de8 -p tcp -m tcp --dport 80 -j ACCEPT
ipv6nat | 2019/11/18 09:37:53 rule added: -t nat -A POSTROUTING 4 -s fd00:beef::3 -d fd00:beef::3 -p tcp -m tcp --dport 80 -j MASQUERADE
ipv6nat | 2019/11/18 09:37:53 rule added: -t nat -A DOCKER 2 -d 0/0 -p tcp -m tcp --dport 80 -j DNAT --to-destination [fd00:beef::3]:80
ipv6nat | 2019/11/18 09:40:09 rule removed: -t filter -D DOCKER -d fd00:beef::3 ! -i br-0d708b1e1de8 -o br-0d708b1e1de8 -p tcp -m tcp --dport 443 -j ACCEPT
ipv6nat | 2019/11/18 09:40:09 rule removed: -t nat -D POSTROUTING -s fd00:beef::3 -d fd00:beef::3 -p tcp -m tcp --dport 443 -j MASQUERADE
ipv6nat | 2019/11/18 09:40:09 rule removed: -t nat -D DOCKER -d 0/0 -p tcp -m tcp --dport 443 -j DNAT --to-destination [fd00:beef::3]:443
ipv6nat | 2019/11/18 09:40:09 rule removed: -t filter -D DOCKER -d fd00:beef::3 ! -i br-0d708b1e1de8 -o br-0d708b1e1de8 -p tcp -m tcp --dport 80 -j ACCEPT
ipv6nat | 2019/11/18 09:40:09 rule removed: -t nat -D POSTROUTING -s fd00:beef::3 -d fd00:beef::3 -p tcp -m tcp --dport 80 -j MASQUERADE
ipv6nat | 2019/11/18 09:40:09 rule removed: -t nat -D DOCKER -d 0/0 -p tcp -m tcp --dport 80 -j DNAT --to-destination [fd00:beef::3]:80
ipv6nat | 2019/11/18 09:40:09 rule removed: -t filter -D DOCKER-ISOLATION-STAGE-1 -i br-0d708b1e1de8 ! -o br-0d708b1e1de8 -j DOCKER-ISOLATION-STAGE-2
ipv6nat | 2019/11/18 09:40:09 rule removed: -t filter -D DOCKER-ISOLATION-STAGE-2 -o br-0d708b1e1de8 -j DROP
ipv6nat | 2019/11/18 09:40:09 rule removed: -t filter -D FORWARD -o br-0d708b1e1de8 -j DOCKER
ipv6nat | 2019/11/18 09:40:09 rule removed: -t filter -D FORWARD -o br-0d708b1e1de8 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
ipv6nat | 2019/11/18 09:40:09 rule removed: -t filter -D FORWARD -i br-0d708b1e1de8 ! -o br-0d708b1e1de8 -j ACCEPT
ipv6nat | 2019/11/18 09:40:09 rule removed: -t filter -D FORWARD -i br-0d708b1e1de8 -o br-0d708b1e1de8 -j ACCEPT
ipv6nat | 2019/11/18 09:40:09 rule removed: -t nat -D POSTROUTING -o br-0d708b1e1de8 -m addrtype --dst-type LOCAL -j MASQUERADE
ipv6nat | 2019/11/18 09:40:09 rule removed: -t nat -D POSTROUTING -s fd00:beef::/80 ! -o br-0d708b1e1de8 -j MASQUERADE
ipv6nat | 2019/11/18 09:40:09 rule removed: -t filter -D FORWARD -j DOCKER-ISOLATION-STAGE-1
ipv6nat | 2019/11/18 09:40:09 rule removed: -t filter -D DOCKER-ISOLATION-STAGE-1 -j RETURN
ipv6nat | 2019/11/18 09:40:09 rule removed: -t filter -D DOCKER-ISOLATION-STAGE-2 -j RETURN
ipv6nat | 2019/11/18 09:40:09 rule removed: -t nat -D PREROUTING -m addrtype --dst-type LOCAL -j DOCKER
ipv6nat | 2019/11/18 09:40:09 rule removed: -t nat -D OUTPUT -m addrtype --dst-type LOCAL -j DOCKER
ipv6nat | # Warning: iptables-legacy tables present, use iptables-legacy to see them
ipv6nat | 2019/11/18 09:40:10 docker-ipv6nat is running in debug mode
ipv6nat | 2019/11/18 09:40:10 rule added: -t filter -A DOCKER-ISOLATION-STAGE-1 1 -j RETURN
ipv6nat | 2019/11/18 09:40:10 rule added: -t filter -A DOCKER-ISOLATION-STAGE-2 1 -j RETURN
ipv6nat | 2019/11/18 09:40:10 rule added: -t nat -A PREROUTING 1 -m addrtype --dst-type LOCAL -j DOCKER
ipv6nat | 2019/11/18 09:40:10 rule added: -t nat -A OUTPUT 1 -m addrtype --dst-type LOCAL -j DOCKER
ipv6nat | 2019/11/18 09:40:11 rule added: -t filter -A FORWARD 1 -j DOCKER-ISOLATION-STAGE-1
ipv6nat | 2019/11/18 09:40:11 rule added: -t filter -A FORWARD 1 -j DOCKER-USER
ipv6nat | 2019/11/18 09:40:11 rule added: -t filter -A FORWARD 3 -o br-0d708b1e1de8 -j DOCKER
ipv6nat | 2019/11/18 09:40:11 rule added: -t filter -A FORWARD 4 -o br-0d708b1e1de8 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
ipv6nat | 2019/11/18 09:40:11 rule added: -t filter -A FORWARD 5 -i br-0d708b1e1de8 ! -o br-0d708b1e1de8 -j ACCEPT
ipv6nat | 2019/11/18 09:40:11 rule added: -t filter -A FORWARD 6 -i br-0d708b1e1de8 -o br-0d708b1e1de8 -j ACCEPT
ipv6nat | 2019/11/18 09:40:11 rule added: -t nat -A POSTROUTING 1 -s fd00:beef::/80 ! -o br-0d708b1e1de8 -j MASQUERADE
ipv6nat | 2019/11/18 09:40:11 rule added: -t nat -A POSTROUTING 1 -o br-0d708b1e1de8 -m addrtype --dst-type LOCAL -j MASQUERADE
ipv6nat | 2019/11/18 09:40:11 rule added: -t filter -A DOCKER-ISOLATION-STAGE-2 1 -o br-0d708b1e1de8 -j DROP
ipv6nat | 2019/11/18 09:40:11 rule added: -t filter -A DOCKER-ISOLATION-STAGE-1 1 -i br-0d708b1e1de8 ! -o br-0d708b1e1de8 -j DOCKER-ISOLATION-STAGE-2
So if I see it correctly, it does add the rules correctly (09:37), then removes them correctly (09:40) on quit (when restarting) but does not add them on the 2nd start (09:40, when restarting).
How did you start it the first time? (09:37) Did you start only ipv6nat at that time? Perhaps it could be some kind of race condition when restarting ipv6nat and caddy at the same time.
Could you also test first restarting caddy, then ipv6nat (instead of restarting them at the same time) and also first ipv6nat, then caddy. Just wondering which cases will end up with the correct rules.
The only difference is in the 1st start I used docker-compose down && docker-compose up -d to destroy whole things and rebuilt them, and in the 2nd I used docker-compose restart.
Caddy should start later than v6nat cause I set depends_on: v6nat for caddy, but not sure if docker will follow this when using restart command.
If I restart them separately, whatever the order are, iptables keeps correct ( with full records include 80&443)
Docker Compose seems don't guarantee the sequence of restart:
If I restart them separately, whatever the order are, iptables keeps correct ( with full records include 80&443)
Actually they are same, whenever I restart v6nat, caddy is running normally at these two situations.
Thanks for the tips. At some point, my X-Forwarded-For started showing a docker IPv4 when the request was made in IPv6 again, as if I just didn't had my ipv6nat container up.
I did a docker-compose down on my docker-file with traefik & ipv6nat, commented out ipv6nat, ran docker-compose up, uncommented ipv6nat, and ran docker-compose up again, and now it works :)
@robbertkl Can we close this?