Macvlan is a lightweight L2 network implementation that does not require traditional bridges. The purpose of this is to have references for plugging into the docker networking APIs which are now available as part of libnetwork. Libnetwork is still under development and is considered as experimental at this point.
- Install the Docker experimental binary from the instructions at: Docker Experimental. (stop other docker instances)
- Quick Experimental Install:
wget -qO- https://experimental.docker.com/ | sh
- Quick Experimental Install:
-
Start Docker with the following. TODO: How to specify the plugin socket without having to pass a bridge name since macvlan/ipvlan dont use traditional bridges. This example is running docker in the foreground so you can see the logs realtime.
docker -d --default-network=macvlan:foo`
-
Download the plugin binary. A pre-compiled x86_64 binary can be downloaded from the binaries directory.
$ wget -O ./macvlan-docker-plugin https://github.com/gopher-net/macvlan-docker-plugin/binaries/macvlan-docker-plugin-0.1-Linux-x86_64 $ chmod +x macvlan-docker-plugin
-
In a new window, start the plugin with the following. Replace the values with the appropriate setup to match the network the docker host is attached to.
$ ./macvlan-docker-plugin \ --gateway=192.168.1.1 \ --macvlan-subnet=192.168.1.0/24 \ --host-interface=eth1 \ -mode=bridge # Or in one line: $ ./macvlan-docker-plugin --gateway=192.168.1.1 --macvlan-subnet=192.168.1.0/24 -mode=bridge --host-interface=eth1
for debugging, or just extra logs from the sausage factory, add the debug flag
./macvlan-docker-plugin -d
. -
Run some containers and verify they can ping one another with
docker run -it --rm busybox
ordocker run -it --rm ubuntu
etc, any other docker images you prefer. Alternatively,docker run -itd busybox
- Or use the script
release-the-whales.sh
in thescripts/
directory to launch a bunch of lightweight busybox instances to see how well the plugin scales for you. It can also serve as temporary integration testing until we get CI setup. See the comments in the script for usage. Keep in mind, the subnet defined incli.go
is the temporarily hardcoded network address192.168.1.0/24
and will hand out addresses starting at192.168.1.2
. This is very temporary until we bind CLI options to the driver data struct.
- Or use the script
-
The default macvlan mode is
bridge
. This works much like traditional vlans where a ToR switch or some other router would be the gateway in a data center. Currently, in order to have two subnets talk to one another it requires an L3 router. This is typical in the vast majority of DCs today. Again, this is just early, more to come and tons of ways to help if interested.
Additional Notes:
- The argument passed to
--default-network
the plugin is identified viamacvlan
. More specifically, the socket file that currently defaults to/usr/share/docker/plugins/macvlan.sock
. If the filepath does not exist at runtime it will be created. - There are default values attached to each flag since a big reason for this project is to help folks learn the libnetwork plugins.
- The containers are brought up on a flat bridge. This means there is no NATing occurring. A layer 2 adjacency such as a VLAN or overlay tunnel is required for multi-host communications. If the traffic needs to be routed an external process to act as a gateway.
- Download a quick macvlan video recording here.
- Since this plugin uses netlink, a Linux host that can build vishvananda/netlink library is required.
Since the network parameters are hardcoded for the next day or two, I think it makes sense to show the output. This will be removed as soon as the network parameters are user defined.
Note: All of the following values can be modified in cli.go
and will be configurable via plugin parameters/config over the next couple of nights of hacking and this section will be removed at that point.
-
The network address is
192.168.1.0/24
and the gateway is192.168.1.1
. The gateway address is an external L3 router (home router, DC L3 leaf node, iptables on a Linux host etc.). -
The hardcoded interface on the docker host is
eth1
(underlying OS that Docker is running on).ip addr show eth1 3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 00:0c:29:c7:a0:3c brd ff:ff:ff:ff:ff:ff inet 192.168.1.254/24 brd 192.168.1.255 scope global eth1
-
Start Docker
docker -d -D --default-network=macvlan:foo
-
Start macvlan-plugin-docker either using
go run main.go
or run the included binary/macvlan-docker-plugin
in the binaries directory. -
Container output (note: since this is a flat single host L2 deployment, the subnet gateway is an external router of
192.168.1.1
):docker run -i -t --rm busybox $ ifconfig eth0 eth0 Link encap:Ethernet HWaddr 72:3A:D8:99:30:A8 inet addr:192.168.1.2 Bcast:0.0.0.0 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:1 errors:0 dropped:0 overruns:0 frame:0 TX packets:7 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:60 (60.0 B) TX bytes:578 (578.0 B) $ ip route default via 192.168.1.1 dev eth0 192.168.1.0/24 dev eth0 proto kernel scope link src 192.168.1.2 / # ping 8.8.8.8 PING 8.8.8.8 (8.8.8.8): 56 data bytes 64 bytes from 8.8.8.8: seq=0 ttl=53 time=23.957 ms ^C --- 8.8.8.8 ping statistics --- 1 packets transmitted, 1 packets received, 0% packet loss round-trip min/avg/max = 23.957/23.957/23.957 ms exit
-
The driver output will look something like the following as a container is created and then removed:
# The following are all the default values. # If you were to only run: go run main.go -d # It would be the same values as below. On almost # all cases you would define a subnet and gateway that # fit into the network you are plugging into. go run main.go -d --gateway=192.168.1.1 --bridge-subnet=192.168.1.0/24 -mode=bridge INFO[0000] Macvlan network driver initialized successfully INFO[0002] The allocated container IP is: [ 192.168.1.2 ] INFO[0002] Created Macvlan port: [ c78c9 ] using the mode: [ bridge ] INFO[0902] Removing unused macvlan link [ c78c9 ] from the removed container
-
As you are hacking, if you run into a scenario where you feel as if you have stale namespaces that are not getting GC'ed you can manually cleanup the netns with the following. An example would be if you
control^c
three times and sent a SIGQUIT rather then waiting on a singlecontrol^c
and theSIGINT
allowing the process to cleaning shutdown.$ umount /var/run/docker/netns/* $ rm /var/run/docker/netns/*
Yes!! This is a purely community project by folks hacking in their free time to provide references for plugging in to Docker via libnetwork remote APIs along with simple deployments for the array of common network architectures deployed in existing data centers today. Please see issues for todos or add todos you would like to add or that we can help you get accomplished if you are new to Docker, Go, networks or programming. This is an excting place and time to evolve with the community. The only rule here is no jerks :)