Create a toxic, connection loss, for Server --> Client ( donwstream)?

Question

Create a toxic, connection loss, for Server --> Client ( donwstream)?

mpet opened this issue a year ago · comments

mpet commented a year ago

Hi,

I have used the toxyiproxy to disable connection between client and server.

client --> server ( I learnt that this is called upstream).

I use the following code to set it up:

        final ToxiproxyClient toxiproxyClient = new ToxiproxyClient(toxiProxy.getHost(), toxiProxy.getControlPort());
        netopeer2Proxy = toxiproxyClient.createProxy("netopeer", "0.0.0.0:" + TOXI_PROXI_PORT,
                "netopeer2:" + NETCONF_SSH_PORT);

When it is setup I try to mimic a connection loss using disconnect/connect using the proxy like this:

       netopeer2Proxy.disable();
        TimeUnit.SECONDS.sleep(DOWNTIME_IN_SECONDS);
        netopeer2Proxy.enable();

This works fine.

Now I want to have the following scenario.

server --> client ( downstream)

Is there a way to do that?

Michael Nikitochkin · Answer 1 · Thu Feb 16 2023 00:38:40 GMT+0800 (China Standard Time)

Hi,

When you disable proxy, it means toxiproxy stops to listen for that port.
If you are looking for different combination how to use Toxiproxy I suggest to check https://github.com/Shopify/toxiproxy/blob/master/scripts/test-e2e

The toxic has an option to specify where to make a problem on Toxiproxy -> Upstream connection or Upstream -> Toxiproxy.
Here is a ruby example: https://github.com/Shopify/toxiproxy#3-using-toxiproxy

mpet · Answer 2 · Thu Feb 16 2023 15:55:34 GMT+0800 (China Standard Time)

@miry I could not find a way to 'cut' connection.

Michael Nikitochkin · Answer 3 · Fri Feb 17 2023 00:10:24 GMT+0800 (China Standard Time)

You can try to use one of the toxics:

mpet · Answer 4 · Mon Feb 20 2023 22:56:27 GMT+0800 (China Standard Time)

I tried this:

I used this for adding a disconnect https://www.testcontainers.org/modules/toxiproxy/

`proxy.toxics().bandwidth("CUT_CONNECTION_DOWNSTREAM", ToxicDirection.DOWNSTREAM, 0);
proxy.toxics().bandwidth("CUT_CONNECTION_UPSTREAM", ToxicDirection.UPSTREAM, 0);

//Try to perform an operation here.

proxy.toxics().get("CUT_CONNECTION_DOWNSTREAM").remove();
proxy.toxics().get("CUT_CONNECTION_UPSTREAM").remove();
`

The thing is that I can setup a client that waits and listen for incoming requests on port 4335.
Then I managed to trigger a "call home" from server to this client using testcontainers.

But problem seems to be the toxic.

NetconfClient <-- Toxic <-- NetconfSever

I have tried different toxics but it seems that they don't apply.

Is there something else I need to consider when I do downstream in java?

br,

//mike

mpet · Answer 5 · Mon Feb 20 2023 23:34:32 GMT+0800 (China Standard Time)

@miry another question. Is there a way to see if data is sent via the proxy created? Could I enable logging?

Michael Nikitochkin · Answer 6 · Mon Feb 20 2023 23:37:02 GMT+0800 (China Standard Time)

@mpet Use env variable LOG_LEVEL=debug to enable detailed logging. It should show connections information.

mpet · Answer 7 · Tue Feb 21 2023 00:06:46 GMT+0800 (China Standard Time)

@miry I tried to set it using VM arg like -DLOG_LEVEL=debug
Unsure what to look for. What should I see? Any examples?

mpet · Answer 8 · Tue Feb 21 2023 17:07:08 GMT+0800 (China Standard Time)

@miry is it possible to see if we go via ToxiProxy.
I use the container:

public ToxiproxyContainer toxiproxy = new ToxiproxyContainer("ghcr.io/shopify/toxiproxy:2.5.0")
    .withNetwork(network);

mpet · Answer 9 · Wed Feb 22 2023 17:26:43 GMT+0800 (China Standard Time)

I also tried setting:

But I cannot really see a difference.

Michael Nikitochkin · Answer 10 · Wed Feb 22 2023 17:59:16 GMT+0800 (China Standard Time)

@mpet I propose to play with toxiproxy-server without Java. Check example how to enable log level trace for toxiproxy-server in https://github.com/Shopify/toxiproxy/blob/master/scripts/test-e2e#L44

mpet · Answer 11 · Thu Feb 23 2023 20:11:45 GMT+0800 (China Standard Time)

@miry I use the docker version. How can I enable debug in it?

mpet · Answer 12 · Mon Feb 27 2023 16:56:18 GMT+0800 (China Standard Time)

I managed to enable debug for the toxiproxy container. This is what I get when it is enabled:

2023-02-27 07:59:45,965 (Wire.java:92) DEBUG : http-outgoing-2 << "[0x1][0x0][0x0][0x0][0x0][0x0][0x0][0xffffffdb]{"level":"info","name":"netconfclient","listen":"toxi:65526","upstream":"host.testcontainers.internal:4335","client":"172.22.0.3:38742","caller":"proxy.go:186","time":"2023-02-27T06:59:45Z","message":"Accepted client"}[\n]"
2023-02-27 07:59:45,965 (Wire.java:92) DEBUG : http-outgoing-4 << "[0x2][0x0][0x0][0x0][0x0][0x0][0x0];[INF]: LN: Trying to connect via IPv4 to 172.22.0.2:65526.[\n]"
2023-02-27 07:59:45,965 (Wire.java:92) DEBUG : http-outgoing-2 << "[\r][\n]"
2023-02-27 07:59:45,965 (Wire.java:92) DEBUG : http-outgoing-4 << "[\r][\n]"
2023-02-27 07:59:45,965 (Slf4jLogConsumer.java:75) INFO : STDERR: [INF]: LN: Trying to connect via IPv4 to 172.22.0.2:65526.
2023-02-27 07:59:45,965 (Slf4jLogConsumer.java:68) INFO : STDOUT: {"level":"info","name":"netconfclient","listen":"toxi:65526","upstream":"host.testcontainers.internal:4335","client":"172.22.0.3:38742","caller":"proxy.go:186","time":"2023-02-27T06:59:45Z","message":"Accepted client"}
2023-02-27 07:59:45,966 (Wire.java:92) DEBUG : http-outgoing-4 << "7f[\r][\n]"
2023-02-27 07:59:45,966 (Wire.java:92) DEBUG : http-outgoing-4 << "[0x2][0x0][0x0][0x0][0x0][0x0][0x0];[INF]: LN: Successfully connected to toxi:65526 over IPv4.[\n]"
2023-02-27 07:59:45,966 (Wire.java:92) DEBUG : http-outgoing-4 << "[0x2][0x0][0x0][0x0][0x0][0x0][0x0]4[INF]: SR: Session 98 (user "root", CID 1) created.[\n]"
2023-02-27 07:59:45,966 (Wire.java:92) DEBUG : http-outgoing-4 << "[\r][\n]"
2023-02-27 07:59:45,966 (Slf4jLogConsumer.java:75) INFO : STDERR: [INF]: LN: Successfully connected to toxi:65526 over IPv4.
2023-02-27 07:59:45,966 (Slf4jLogConsumer.java:75) INFO : STDERR: [INF]: SR: Session 98 (user "root", CID 1) created.
2023-02-27 07:59:45,966 (Wire.java:92) DEBUG : http-outgoing-4 << "79[\r][\n]"
2023-02-27 07:59:45,966 (Wire.java:92) DEBUG : http-outgoing-4 << "[0x2][0x0][0x0][0x0][0x0][0x0][0x0]4[INF]: SR: Session 99 (user "root", CID 1) created.[\n]"
2023-02-27 07:59:45,966 (Wire.java:92) DEBUG : http-outgoing-4 << "[0x2][0x0][0x0][0x0][0x0][0x0][0x0]5[INF]: SR: Session 100 (user "root", CID 1) created.[\n]"
2023-02-27 07:59:45,966 (Wire.java:92) DEBUG : http-outgoing-4 << "[\r][\n]"
2023-02-27 07:59:45,966 (Slf4jLogConsumer.java:75) INFO : STDERR: [INF]: SR: Session 99 (user "root", CID 1) created.
2023-02-27 07:59:45,966 (Slf4jLogConsumer.java:75) INFO : STDERR: [INF]: SR: Session 100 (user "root", CID 1) created.
2023-02-27 07:59:45,987 (Wire.java:92) DEBUG : http-outgoing-2 << "151[\r][\n]"
2023-02-27 07:59:45,987 (Wire.java:92) DEBUG : http-outgoing-2 << "[0x1][0x0][0x0][0x0][0x0][0x0][0x1]I{"level":"error","name":"netconfclient","listen":"toxi:65526","upstream":"host.testcontainers.internal:4335","error":"dial tcp: lookup host.testcontainers.internal on 127.0.0.11:53: no such host","client":"172.22.0.3:38742","caller":"proxy.go:193","time":"2023-02-27T06:59:45Z","message":"Unable to open connection to upstream"}[\n]"
2023-02-27 07:59:45,987 (Wire.java:92) DEBUG : http-outgoing-2 << "[\r][\n]"
2023-02-27 07:59:45,987 (Slf4jLogConsumer.java:68) INFO : STDOUT: {"level":"error","name":"netconfclient","listen":"toxi:65526","upstream":"host.testcontainers.internal:4335","error":"dial tcp: lookup host.testcontainers.internal on 127.0.0.11:53: no such host","client":"172.22.0.3:38742","caller":"proxy.go:193","time":"2023-02-27T06:59:45Z","message":"Unable to open connection to upstream"}
2023-02-27 07:59:45,988 (Wire.java:92) DEBUG : http-outgoing-4 << "30[\r][\n]"
2023-02-27 07:59:45,988 (Wire.java:92) DEBUG : http-outgoing-4 << "[0x2][0x0][0x0][0x0][0x0][0x0][0x0]([ERR]: LN: SSL_accept failed (Success).[\n]"
2023-02-27 07:59:45,988 (Wire.java:92) DEBUG : http-outgoing-4 << "[\r][\n]"
2023-02-27 07:59:45,988 (Slf4jLogConsumer.java:75) INFO : STDERR: [ERR]: LN: SSL_accept failed (Success).

What is your interpretation of this?

//mike

Michael Nikitochkin · Answer 13 · Mon Feb 27 2023 23:38:05 GMT+0800 (China Standard Time)

I see there is a mix of java app logs and toxiproxy-server.

There is an error, that says there is no upstream host:

{"level":"error","name":"netconfclient","listen":"toxi:65526","upstream":"host.testcontainers.internal:4335","error":"dial tcp: lookup host.testcontainers.internal on 127.0.0.11:53: no such host","client":"172.22.0.3:38742","caller":"proxy.go:193","time":"2023-02-27T06:59:45Z","message":"Unable to open connection to upstream"}

In those logs, I don't see any DEBUG level messages.

My suggestion is the same:

Get some knowledge of "how it works" and "how to add toxics" with README.
Play with toxyproxy-server and toxiproxy-cli without Testcontainers or other wrappers.

It would help you later to understand logs from Testcontainers.
If you still struggle with configuration toxiproxy,
I suggest asking reddit or stackoverflow for help.
We use github issues only for feature requests and bug reports.

In case you find an implementation problem of Toxiproxy let me know
by creating a new issue. Also, we are a welcome contribution to our docs.