Possibility of getting this working with nftables

[Sven Grunewaldt]

I've been using docker-ipv6nat for a while on my old server and am pretty happy with it. On a new server I intended to use nftables, which is basically working fine, using this guide.

The problem I have is that I can't use static assigned IP addresses, since this isn't supported in docker-compose. (at least in version 3 files, version 2 apparently works)
This kind of forces me to fall back to "the old way" and let Docker handle the IP assignment, but then I can't put in static nftables rules for the exposed containers.

I'm fine with running this workaround again, but it obviously only supports iptables. I don't really know Go and a quick check of the code made my head spin, so I'd like to use this issue as a discussion if nftables support is even possible for this project.

The only thing that needs to happen is to create a forward rule when a container comes up, nothing else. The rest of the setup is already handled in the configured nftables config (as far as I can see). This is pretty much the same thing docker-ipv6nat is already doing, just with other commands.

There are iptables-compat tools that use the old iptables syntax but creates nftables rules, but I don't know if this could work with docker-ipv6nat.

[Robbert Klarenbeek]

I'd like to keep docker-ipv6nat as close to Docker as possible, as far as functionality. I'm not sure if Docker supports nftables and I don't have any experience with nftables myself.

If you could replace your ip6tables command with a wrapper that uses iptables-compat, shouldn't it work, without modifying docker-ipv6nat?

[Sven Grunewaldt]

I'm not sure if Docker supports nftables and I don't have any experience with nftables myself.

It actually doesn't (moby/moby#26824). While thinking a bit more about this I realized that docker-ipv6nat would have to set IPv4 forward rules in this case too, since you (have to?) disable iptables in Docker completely so it doesn't mess with nftables.

If you could replace your ip6tables command with a wrapper that uses iptables-compat, shouldn't it work, without modifying docker-ipv6nat?

After creating the issue I checked out the go-iptables library docker-ipv6nat is using and realized it simply calls iptables instead of using some API. I already played around with a special debian Docker container that replaces iptables with iptables-nftables-compat, but didn't put much more time into it due to the general missing nftables support described above.

I guess for docker-ipv6nat it only makes sense to wait for Docker itself to support nftables and then implement support for it, in case Docker doesn't include some kind of IPv6 NAT itself then. (moby/libnetwork#2023)

Especially considering that the guide I linked in the original issue is actually about disabling the Docker NAT magic and controlling it manually – kind of defeats the purpose getting docker-ipv6nat in the mix. The main problem is docker-compose not supporting static IPs in compose file version 3+.

Edit:
I finally convinced docker-compose to use static IPs. The issue was that one has to create the NAT network manually with docker network create (there is an example in the guide I linked) and then use this network with external: true in the docker-compose.yml.

Example:

docker network create --ipv6 --subnet=172.18.0.0/16 --subnet=fcdd:1::/48 ipv6-nat

version: "3.1"

networks:
  ipv6-nat:
    external: true

services:
  app:
    […]
    networks:
      ipv6-nat:
        ipv4_address: 172.18.50.1
        ipv6_address: fcdd:1::50:1

Not relevant to this issue per se, so please excuse the noise… may be helpful for people stumbling over this with Google or skimming through the issues.

[Denis Denisov]

nftables is also no longer considered a modern solution, with the advent of bpfilter https://www.mail-archive.com/[e-mail]/msg11127.html

[Robbert Klarenbeek]

Safe to close this issue then?

[Sven Grunewaldt]

Eh, I don't think so? At least not for this reason. nftables is stable and ready to use in Debian Stretch. Who knows when bpfilter will turn up in stable distributions. I mean, just read the first sentence in the linked mailing list thread:

This is a very rough and early proof of concept that implements bpfilter.

LWN has an article on it dated Feb 19 that states in conclusion:

The end result is that we'll probably not see bpfilter in the mainline kernel in the immediate future. Given the developers who have worked on it, though, bpfilter is clearly a serious initiative that is firmly aimed at getting into the mainline eventually. If it truly proves to be a better solution to the network packet-filtering problem, those developers seem likely to prevail eventually.

Regarding closing this issue: I'm usually against having open issues that only document a specific edge case since wiki entries are in most cases superior in those situations, but this issue nonetheless may help people researching how to get IPv6 working in Docker. It does that while being closed too, though. So I'd say go ahead, close the issue, but for other reasons than bpfilter ;)

I've been running the "manual" nftables setup for a while now, by the way. Works really great, though having to add each new subnet to the nftables rules can get tedious, otherwise containers in new networks won't be able to connect to the outside world. Doesn't happen often on my server and is just a quick edit of the nftables.conf, so I'm completely fine with it.

[Robbert Klarenbeek]

Thanks for the extended explanation! I'll close this then, since it's out-of-scope for this project.