lukaszlach / docker-tc

:whale: :vertical_traffic_light: Docker Traffic Control - network rate limiting, emulating delays, losses, duplicates, corrupts and reorders of network packets using only container labels or a command-line interface.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

docker-tc configures settings on the wrong veth's

grypyrg opened this issue · comments

For a handson tutorial at a conference (MySQLDays, a FOSDEM Fringe conf, presenting about MySQL InnoDB Cluster/ReplicaSet), I was using docker-tc to simulate network issues.
However, during the class, I found that weird things happened when I tried to network partition a single host. Other hosts seemed to be affected too.

in the examples I have a docker-tc wrapper that contains:

$ curl -sSf -X "$1" "localhost:4080/$2?$3"

Here's the state of the containers before doing anything:

$ docker-tc list
# id=e1b59d438067 name=mysqltutorial_router_3
qdisc noqueue 0: root refcnt 2 
# id=fce32b583115 name=mysqltutorial_router_2
qdisc noqueue 0: root refcnt 2 
# id=0f16b00c62a9 name=mysqltutorial_router_5
qdisc noqueue 0: root refcnt 2 
# id=21bf74c760fd name=mysqltutorial_router_4
qdisc noqueue 0: root refcnt 2 
# id=1b491649ba7f name=mysqltutorial_router_1
qdisc noqueue 0: root refcnt 2 
# id=044954e67ea2 name=mysqltutorial_app_run_1ce5130790c3
qdisc noqueue 0: root refcnt 2 
# id=898e634ecbeb name=mysqltutorial_mysqlsh_run_8882df05c8be
qdisc noqueue 0: root refcnt 2 
# id=acc4be45b9bd name=mysqltutorial_server_1_1
qdisc noqueue 0: root refcnt 2 
# id=765692e6d3cf name=mysqltutorial_server_2_1
qdisc noqueue 0: root refcnt 2 
# id=7cd13e68ff62 name=mysqltutorial_server_3_1
qdisc noqueue 0: root refcnt 2 
qdisc noqueue 0: root refcnt 2 
qdisc noqueue 0: root refcnt 2 
qdisc noqueue 0: root refcnt 2 
qdisc noqueue 0: root refcnt 2 
# id=308522eccfac name=mysqltutorial_registrator_1
qdisc noqueue 0: root refcnt 2 
qdisc noqueue 0: root refcnt 2 
qdisc noqueue 0: root refcnt 2 
# id=1e8d4ce2351e name=docker-tc
# id=582876fe7a4b name=mysqltutorial_consul_1
qdisc noqueue 0: root refcnt 2 

Then I network partition a single container by adding loss=100%:

$ docker-tc post mysqltutorial_server_3_1 loss=100%
Set loss=100% on veth3795593
Controlling traffic of the container /mysqltutorial_server_3_1 on veth3795593
Set loss=100% on veth6015f81
Controlling traffic of the container /mysqltutorial_server_3_1 on veth6015f81
Set loss=100% on vethfeda243
Controlling traffic of the container /mysqltutorial_server_3_1 on vethfeda243
Set loss=100% on vethd9de0b8
Controlling traffic of the container /mysqltutorial_server_3_1 on vethd9de0b8
Set loss=100% on veth23ae4cf
Controlling traffic of the container /mysqltutorial_server_3_1 on veth23ae4cf

When I then look at the configuration, you can see that various other servers now also have loss=100%:

$ docker-tc list
# id=e1b59d438067 name=mysqltutorial_router_3
qdisc netem 1: root refcnt 2 limit 1000 loss 100%
# id=fce32b583115 name=mysqltutorial_router_2
qdisc netem 1: root refcnt 2 limit 1000 loss 100%
# id=0f16b00c62a9 name=mysqltutorial_router_5
qdisc netem 1: root refcnt 2 limit 1000 loss 100%
# id=21bf74c760fd name=mysqltutorial_router_4
qdisc noqueue 0: root refcnt 2 
# id=1b491649ba7f name=mysqltutorial_router_1
qdisc noqueue 0: root refcnt 2 
# id=044954e67ea2 name=mysqltutorial_app_run_1ce5130790c3
qdisc noqueue 0: root refcnt 2 
# id=898e634ecbeb name=mysqltutorial_mysqlsh_run_8882df05c8be
qdisc netem 1: root refcnt 2 limit 1000 loss 100%
# id=acc4be45b9bd name=mysqltutorial_server_1_1
qdisc noqueue 0: root refcnt 2 
# id=765692e6d3cf name=mysqltutorial_server_2_1
qdisc noqueue 0: root refcnt 2 
# id=7cd13e68ff62 name=mysqltutorial_server_3_1
qdisc netem 1: root refcnt 2 limit 1000 loss 100%
qdisc netem 1: root refcnt 2 limit 1000 loss 100%
qdisc netem 1: root refcnt 2 limit 1000 loss 100%
qdisc netem 1: root refcnt 2 limit 1000 loss 100%
qdisc netem 1: root refcnt 2 limit 1000 loss 100%
# id=308522eccfac name=mysqltutorial_registrator_1
qdisc noqueue 0: root refcnt 2 
qdisc noqueue 0: root refcnt 2 
qdisc netem 1: root refcnt 2 limit 1000 loss 100%
# id=1e8d4ce2351e name=docker-tc
# id=582876fe7a4b name=mysqltutorial_consul_1
qdisc noqueue 0: root refcnt 2