docker-tc configures settings on the wrong veth's
grypyrg opened this issue · comments
For a handson tutorial at a conference (MySQLDays, a FOSDEM Fringe conf, presenting about MySQL InnoDB Cluster/ReplicaSet), I was using docker-tc
to simulate network issues.
However, during the class, I found that weird things happened when I tried to network partition a single host. Other hosts seemed to be affected too.
in the examples I have a docker-tc
wrapper that contains:
$ curl -sSf -X "$1" "localhost:4080/$2?$3"
Here's the state of the containers before doing anything:
$ docker-tc list
# id=e1b59d438067 name=mysqltutorial_router_3
qdisc noqueue 0: root refcnt 2
# id=fce32b583115 name=mysqltutorial_router_2
qdisc noqueue 0: root refcnt 2
# id=0f16b00c62a9 name=mysqltutorial_router_5
qdisc noqueue 0: root refcnt 2
# id=21bf74c760fd name=mysqltutorial_router_4
qdisc noqueue 0: root refcnt 2
# id=1b491649ba7f name=mysqltutorial_router_1
qdisc noqueue 0: root refcnt 2
# id=044954e67ea2 name=mysqltutorial_app_run_1ce5130790c3
qdisc noqueue 0: root refcnt 2
# id=898e634ecbeb name=mysqltutorial_mysqlsh_run_8882df05c8be
qdisc noqueue 0: root refcnt 2
# id=acc4be45b9bd name=mysqltutorial_server_1_1
qdisc noqueue 0: root refcnt 2
# id=765692e6d3cf name=mysqltutorial_server_2_1
qdisc noqueue 0: root refcnt 2
# id=7cd13e68ff62 name=mysqltutorial_server_3_1
qdisc noqueue 0: root refcnt 2
qdisc noqueue 0: root refcnt 2
qdisc noqueue 0: root refcnt 2
qdisc noqueue 0: root refcnt 2
qdisc noqueue 0: root refcnt 2
# id=308522eccfac name=mysqltutorial_registrator_1
qdisc noqueue 0: root refcnt 2
qdisc noqueue 0: root refcnt 2
qdisc noqueue 0: root refcnt 2
# id=1e8d4ce2351e name=docker-tc
# id=582876fe7a4b name=mysqltutorial_consul_1
qdisc noqueue 0: root refcnt 2
Then I network partition a single container by adding loss=100%
:
$ docker-tc post mysqltutorial_server_3_1 loss=100%
Set loss=100% on veth3795593
Controlling traffic of the container /mysqltutorial_server_3_1 on veth3795593
Set loss=100% on veth6015f81
Controlling traffic of the container /mysqltutorial_server_3_1 on veth6015f81
Set loss=100% on vethfeda243
Controlling traffic of the container /mysqltutorial_server_3_1 on vethfeda243
Set loss=100% on vethd9de0b8
Controlling traffic of the container /mysqltutorial_server_3_1 on vethd9de0b8
Set loss=100% on veth23ae4cf
Controlling traffic of the container /mysqltutorial_server_3_1 on veth23ae4cf
When I then look at the configuration, you can see that various other servers now also have loss=100%
:
$ docker-tc list
# id=e1b59d438067 name=mysqltutorial_router_3
qdisc netem 1: root refcnt 2 limit 1000 loss 100%
# id=fce32b583115 name=mysqltutorial_router_2
qdisc netem 1: root refcnt 2 limit 1000 loss 100%
# id=0f16b00c62a9 name=mysqltutorial_router_5
qdisc netem 1: root refcnt 2 limit 1000 loss 100%
# id=21bf74c760fd name=mysqltutorial_router_4
qdisc noqueue 0: root refcnt 2
# id=1b491649ba7f name=mysqltutorial_router_1
qdisc noqueue 0: root refcnt 2
# id=044954e67ea2 name=mysqltutorial_app_run_1ce5130790c3
qdisc noqueue 0: root refcnt 2
# id=898e634ecbeb name=mysqltutorial_mysqlsh_run_8882df05c8be
qdisc netem 1: root refcnt 2 limit 1000 loss 100%
# id=acc4be45b9bd name=mysqltutorial_server_1_1
qdisc noqueue 0: root refcnt 2
# id=765692e6d3cf name=mysqltutorial_server_2_1
qdisc noqueue 0: root refcnt 2
# id=7cd13e68ff62 name=mysqltutorial_server_3_1
qdisc netem 1: root refcnt 2 limit 1000 loss 100%
qdisc netem 1: root refcnt 2 limit 1000 loss 100%
qdisc netem 1: root refcnt 2 limit 1000 loss 100%
qdisc netem 1: root refcnt 2 limit 1000 loss 100%
qdisc netem 1: root refcnt 2 limit 1000 loss 100%
# id=308522eccfac name=mysqltutorial_registrator_1
qdisc noqueue 0: root refcnt 2
qdisc noqueue 0: root refcnt 2
qdisc netem 1: root refcnt 2 limit 1000 loss 100%
# id=1e8d4ce2351e name=docker-tc
# id=582876fe7a4b name=mysqltutorial_consul_1
qdisc noqueue 0: root refcnt 2