Fix IPv6 for docker links
opened this issue · comments
Please fix IPv6 docker links, where incorrectly only IPv4 container addresses appear to be set even with IPv6 enabled in the docker daemon.
Respective docker bug ticket: moby/moby#17666
The issue is with the userland proxy, which is proxying the connections to the containers. I believe this userland proxy is on its way out, as there are already efforts on disabling it by default (moby/moby#14856). However, disabling the userland proxy (add --userland-proxy=false to your docker daemon startup arguments) will break published ports for IPv6 completely, whether you have IPv6 enabled containers (docker daemon with --ipv6) or not.
IPv4 published ports will still keep on working without the userland proxy, because of NAT, which is also why it shows the correct remote IPv4 addresses on incoming connections.
So yes, just like all those other IPv6 issues (I feel your pain @Jonast), published ports are quite messed up with IPv6. To make it consistent with IPv4, probably NAT will have to be used, and lots of people seem to be against this. If there are better alternatives I'm all for skipping NAT, but right now (published ports being 'broken') I believe NAT for IPv6 published ports could finally fix some issues:
- Published ports would work as expected (container seeing the correct remote IPv6)
- Containers will not have a publicly accessible IPv6 address, and it's ports won't be publicly accessible (only its published ports)
- The much criticised userland proxy can be disabled / removed
- Consistency between IPv4 and IPv6
Of course, a use case for containers with publicly accessible IPv6 addresses exists as well, but that's completely different from how most people are using (IPv4) containers right now.
Just to clarify, I'm not for NAT per se. If there are more fitting alternatives that solve the above issues: even better! I just believe NAT seems to work for IPv4 right now: people are used to having their containers tucked away behind a NAT layer which protects the container from public access, are used to published ports for opening them up to the outside world. Container images / setups are built around this idea.
Having it work exactly the same for IPv6 would make the global transition to IPv6 much smoother.
For what it's worth: I would suggest NAT is actually inferior, because you can't combine some services directly reachable by a global IPv6 container IP, and other services reachable through the main IP. Simply keeping the userland proxy seems like a better idea.
However just to clarify, I really don't care much how this is done and whether it uses NAT or not. But publish for IPv6 should work properly, people rely on it. Or if you want it to be removed, figure out some other consistent solution for both IPv4/IPv6 and add it soon so people can figure out some workflow that works properly no matter what Internet Protocol - I can't have hidden backend services on IPv4 proxied through the front that are suddenly directly reachable through IPv6, that's just a security and inconsistency nightmare.
Also if not choosing NAT, then EXPOSE still must not ever expose the port to the outside world per default. There should be another extra option added like local publish. Or, again, make it behave that way with IPv4 too. Making such a vital central feature inconsistent is going to trip people up in really nasty ways.
To me, services directly reachable by a global IPv6 container IP vs. services reachable through the main IP are 2 different use cases, which would require different (explicit) configuration/arguments. This also forces you to think about what you're doing, for example creating a publicly available container.
But why wouldn't you be able to combine those 2 if NAT were to be used?
On keeping the userland proxy: besides hiding the original remote IP, I've seen various other complaints about the userland proxy, e.g. being very resource intensive and not performing well enough in terms of network speed/throughput. Perhaps it's just poorly written and needs to be replaced with a new one, but there seems to be a growing tendency to disable it completely.
Please note: if you have no public IPv6 address space to give each of your containers a public IPv6, or you don't need/want it (e.g. you want everything proxied through 1 main IPv6 of the host), you can run docker without --ipv6 and --fixed-cidr-v6 and it will work fine, as long as the userland proxy is enabled, which is the default. Exposed ports will work on both the host's main IPv4 and IPv6 addresses and your containers will NOT be accessible publicly, since they don't have their own (public) IPv6. In that case, the only remaining problem (aside from the above flaws of the userland proxy) is the rewritten remote IP.
To me, services directly reachable by a global IPv6 container IP vs. services reachable through the main IP are 2 different use cases, which would require different (explicit) configuration/arguments. This also forces you to think about what you're doing, for example creating a publicly available container.
Exactly, which is why either the userland proxy or NAT should be used for IPv6 to allow for both use cases. Hopefully, others will share this opinion so IPv6 publish can be finally fixed.
But why wouldn't you be able to combine those 2 if NAT were to be used?
Please note I'm not a NAT expert. Maybe that remark was misguided. I just assumed if my container IPs are behind a NAT they won't be globally directly reachable IPs on their own, which would make the containers not globally reachable. However I guess if they have multiple IPs... but then again that seems a bit overcomplicated unlike the userland proxy which is imho actually a quite simple and elegant solution. But again, if NAT works fine I don't mind that either.
In that case, the only remaining problem (aside from the above flaws of the userland proxy) is the rewritten remote IP.
That is however, a really huge problem. You can't make safe we bservices without any possibility of source IP knowledge. This starts with security vital things as forum sessions that expire properly on user IP change, and goes to practical problems like banning spammers. This is why I opened this bug report. It really needs to be fixed.
Also that docker links simply ignore IPv6 is a really annoying and at the same time seemingly low-hanging-fruit problem. It would go a long way if it could be fixed, while it doesn't seem like something that would be hard to do.
Please see docker-ipv6nat for a working solution for IPv6 published ports. It runs as a container and extends Docker with IPv6 NAT, which just works (the same way it works for IPv4).
Linking is a different issue, but I don't mind that my containers communicate internally over IPv4 for the time being. What I care about is exposing my services to the outside world on IPv6, which works perfectly when using docker-ipv6nat.
I removed the mention of -p/--publish from this ticket, since there is already a separate ticket for this here: moby/moby#17666 , and a solution proposed here moby/moby#25407 (where IPv6 NAT can be discussed further).
As for the docker links lacking IPv6 container addresses, does anyone know the current state of this? Has this been fixed? (It sounds like it would be fairly easy to fix, but maybe I'm wrong on this)
This issue is somewhat a duplicate of moby/moby#13228 so I'm closing it for now.