Use new isolation scheme
bephinix opened this issue · comments
Docker now uses DOCKER-ISOLATION-STAGE-1 and DOCKER-ISOLATION-STAGE-2 to isolate traffic on bridges. As we want to mirror Docker, we should also move to the new isolation scheme.
/cc @robbertkl
Sorry if this is a bad time to bring this up, but this just reminded me that this exists and just sits there: moby/libnetwork#2023 - given docker folks can be somehow convinced to finally merge this, it might make it unnecessary to constantly update docker-ipv6nat to keep up with things. Not sure that will ever happen though
@Jonast Yes, I'm aware. I'm subscribed to the same issue, but I'm getting the idea it will never get merged. I wish it will, though: I'd also rather have this built-in to docker itself.
@bephinix Thanks for reporting, this is news for me. Not sure when they added this, but my Docker for Mac install already has this (18.06.1-ce). You're right, this should be mirrored in ipv6nat as well.
I guess it's finally time to ask the docker daemon what version it's running and do different things, based on the docker version, to make it stay compatible (mirrored) with older versions.
@Jonast You are absolutely right, but we need a nearly perfect workaround solution until the merge is complete.
@robbertkl Can you estimate how hard it is to mirror this new isolation scheme?
@robbertkl
Simply start a VM (debian) and install docker-ce.
/etc/docker/daemon.json:
{
"bridge": "none",
"userland-proxy": false
}
Restart and run:
docker network create -d bridge --ipv6 --subnet 172.31.0.0/24 --subnet fdef::/64 -o "com.docker.network.bridge.enable_icc=false" -o "com.docker.network.bridge.name=testnet" testnet
You will see the differences in "-t nat" and "-t filter" of iptables/ip6tables.
Yeah, I've already seen it on my Docker for Mac instance. Can't give you an estimate yet, have to look into it but unfortunately not much free time at this moment.
Any update on this?
No progress, sorry..
It was implemented by this MR especially this commit.
This was done to reduce startup time for docker as many iptables rules were generated. WIth the mentioned MR the complexity was reduced from O(2^n) to O(2n), as you can read here and here.
New Isolation Scheme
We need to create two chains (STAGE-1 and STAGE-2) and jump from FORWARD to STAGE-1.
For every bridge network which is internal, the following rules are created in STAGE-1:
DROP all -- * dckrIstInternal !172.25.1.0/24 0.0.0.0/0
DROP all -- dckrIstInternal * 0.0.0.0/0 !172.25.1.0/24
For every bridge network which is not internal, the following rule is created in STAGE-1:
DOCKER-ISOLATION-STAGE-2 all -- dckrIsNotInternal !dckrIsNotInternal 0.0.0.0/0 0.0.0.0/0
In addition to that, for every bridge network which is not internal, the following rule is created in STAGE-2:
DROP all -- * dckrIsNotInternal 0.0.0.0/0 0.0.0.0/0
This will provide the bridge isolation.
After the isolation, ICC and external access is checked in FORWARD rules, but this was not changed by mentioned MR.
Fun-Fact: Current network handling is causing some problems with external ports, see here.
Summary
Stage 1
If a packet from an internal network should be forwarded to another bridge or if a packet from another bridge should be forwarded to an internal network, this packet will be dropped.
If a packet from an non-internal network should be forwarded to another bridge, Stage-2 will be entered.
Stage 2
All packets entering this stage come from an docker network and should leave this network.
In this stage, it will be checked if the packets should be forwarded to another docker network.
If this is the case, it will be dropped.
TL;DR
Stage 1 will ensure that packets cannot leave internal networks and that external packets cannot enter internal networks.
Stage 1 and 2 together will ensure, that packets from a docker network cannot be forwarded to another docker network.
@robbertkl Do you know when you will be able to review the open MRs?
Just merged all of them, sorry for the delay.