powerfulseal / powerfulseal

A powerful testing tool for Kubernetes clusters.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Affect network traffic after startup

Aergonus opened this issue · comments

Currently the network tc-clone only allows an init container that will setup an affected environment. If we want to simulate a case where an application can execute startup connections but then get delayed queries (i.e. after establishing a database connection or receiving leader election results) this is not possible. This is easily possible by adding a sidecar container (this may break some automation in large scale production environments). However, a more complicated approach such as dynamically updating network attacks (delay for one minute then fully drop traffic) requires a more dynamic approach. It seems like kubernetes jobs is not able to do this and creating a listening server for commands is overkill.

The simple sidecar vs init container forces the user to do a lot of the heavy-lifting. Exposing a wait interface with a delay could be beneficial. This would look something like

    delay = max(spec.get("delay", 0), 0)  # used for initialDelaySeconds
    if delay == 0:  # default init for backwards compatibility
 ...
           # initialDelaySeconds=delay,

For a real use case, this container should probably have some kind of wait_for script and expose configurations for configuring the native probes for a container.

regarding the user side heavy lifting there are some unanswered questions on the best way to deal with the sidecar chaos injectors. Without a properly crafted command to stay up and awake, they will finish and reboot loop. This means either the command must be idempotent or the container must essentially hang itself while staying alive for probes.