Sending a SyncRequest from within a SyncRequest and getting a response

[Matt]

Hey!

I'm in a situation where I would need this to work:

1. Client sends a SyncRequest to Server
2. Server requires more info and therefore sends a SyncRequest back to Client
3. Client replies with a SyncResponse including the requested info
4. Server replies to the original SyncRequest (from step 1)

Basically: client -> server (extra: -> client -> server) -> client, if that makes sense.

To explain it better, here is some code to illustrate what I'd want to do (fiddle):

using System.Text;
using WatsonTcp;

const string host = "127.0.0.1";
const int port = 1337;

var server = new WatsonTcpServer(host, port);
var client = new WatsonTcpClient(host, port);

server.Callbacks.SyncRequestReceived += request =>
{
    Console.WriteLine(2);
    var response = server.SendAndWait(1000, request.IpPort, "hello from server!");
    
    Console.WriteLine(4);
    return new SyncResponse(request, response.Data);
};

client.Callbacks.SyncRequestReceived += request =>
{
    Console.WriteLine(3);
    return new SyncResponse(request, "hello back from client!");
};

server.Events.MessageReceived += (_, _) => { };
client.Events.MessageReceived += (_, _) => { };

server.Start();
client.Connect();

Console.WriteLine(1);
var response = client.SendAndWait(1000, "hello from client!");

Console.WriteLine("client received response: " + Encoding.UTF8.GetString(response.Data));
Console.ReadKey(true);

However, this is the output I am getting from this exact code:

1
2
3
Unhandled exception. System.TimeoutException: A response to a synchronous request was not received within the timeout window.

As you can see, the client does not seem to be sending a response back, or at least the server isn't letting me know about it (stuck at step 3). If I was to guess, and without having read the entire codebase, I would assume this is the intended behavior and is due to a lock() or SemaphoreSlim somewhere, preventing concurrent requests/responses?

This would make sense as otherwise, it would mean multiple different data could be received concurrently, mixing everything up. Hopefully I'm not saying sh*t here. Please, correct me if so 😅 Always looking to get more knowledge!

Considering I am right (and even if I am not), then how would I go about getting this to work?

What I thought of is: Instead of sending a SyncRequest back from within a SyncRequest, what I could do is make the server reply right away with a SyncResponse saying something along the lines of "hey! more data is required! please come back with a new SyncRequest containing it!", but that is pretty inconvenient and would not work well with my current structure. My question here really is about (issue title): "Sending a SyncRequest from within a SyncRequest and getting a response", without getting blocked.

So, what can/should I do? Is that even possible at all or should I completely forget about it and instead go with the idea above?

[Joel Christner]

Seems like getting SendAndWait to work might solve the issue, but, it's not the approach I would take to address what you're describing. It sounds like what you want is a state machine. Personally, my preference is to maintain state variables and just use async message handlers rather than using SendAndWait. In the main message handler, have it check the state variables to determine what state the machine is in, and, handle the message accordingly.

I added a comment to the other issue on SendAndWait, let's see what comes back from that, it could be very telling.

[Matt]

Ok, sure, let's solve one issue at a time. It might get quickly distracting otherwise.

Seems like getting SendAndWait to work might solve the issue

Really not sure about that though, both the shown code and the fiddle are running on latest v4.

[Joel Christner]

So usually in situations like this I have an enum that defines the various states that the state machine could be in, for instance:

• NotAuthenticated
• Authenticated
• SessionSetupRequested
• Normal
  etc (these are just examples)

And I hold this as a variable that is accessible to the entirety of the entity (for instance, as a private variable in a class instance). Then, the handler for MessageReceived would first consult this enum to determine what the state is, and, handle the incoming message accordingly.

Doing this with SendAndWait is certainly doable, but I'd strongly advise against it. First, using sync send-and-wait approaches is very, very error prone when you have nested send-and-wait operations, and, handling the variety of failure cases creates a lot of work and it's highly unlikely that you'll catch everything. Second, you can accomplish the same with state variables and async messages which require no nesting of handlers.

Let me know what you think.

The state machine I use as a reference: http://tcpipguide.com/free/t_TCPOperationalOverviewandtheTCPFiniteStateMachineF-2.htm (I'm sure you're already familiar with TCP and state machines, but sharing it here on the off chance you aren't)

Cheers, Joel

[Matt]

Thanks for your answer!

Oh well, I have to say I am not so familiar with TCP and/or state machines. I have a bunch of skills, but sadly those aren't part of it 😅 I know how TCP works, in theory, but I am not so familiar with the practice. Let's just say bytes and streams aren't my thing hahah, at least for now.

Anyway, enough about me.

Let's be honest, I don't quite understand a big part of your answer. This is probably because of my lack of skill related the topics mentioned above, but that is fine. I'll "let you know what I think" by explaining what all of this is about in the first place.

I do not understand why you seem so against recommending SendAndWait. I know you tried to explain it, but I don't quite get your point. I mostly understand what you are saying about the error-handling stuff, but that's basically it.

To give you more context about my original problem: I've been working on a wrapper around WatsonTcp, which is mostly about the ability to send/receive what I call "commands", between one server and many clients. This is something I've been needing, hence why I'm making it.

For even more context, here's an example of how a working (callback, which returns something) command looks like:

public class EchoTcpCommand : TcpCallbackCommand<EchoTcpCommand, string>
{
    public string Text { get; }

    public EchoTcpCommand(string text) : base(TimeSpan.FromSeconds(5))
    {
        Text = text;
    }

    internal override async Task<string> ExecuteAsync(TcpCommandContext context)
    {
        return Text;
    }
}

Internally, this uses SendAndWait, with proper error handling (I'd like to believe). As you can see, it is pretty simple and straightforward. This command can then be used like that:

string response = server.SendCallbackCommandToClient<EchoTcpCommand, string>(client, new EchoTcpCommand("Hello!"));

So yeah, that's the main feature of the wrapper I'm working on.

Now, to be more precise about this whole issue, this is what I'd like to be able to do:

internal override async Task ExecuteAsync(TcpCommandContext context)
{
    if (context.Sender.IsServer)
    {
        // one-way (non-callback) command, not sure if that would block too since it uses "SendAsync" and not "SendAndWait", but that is just for the example anyway.
        await context.Receiver.As<AeroClientBase>().SendCommandToServerAsync(new WhateverTcpCommand());
    }
}

Sadly, this is not possible, as you have understood by now. The code is valid, and I would have expected it to work, but it doesn't. Because of this "blocking" problem.

From there however, I'm not sure which path to take. There are a few workarounds I was able to think of (one of them being mentioned in one of my replies above), but none are optimal enough and/or would be as easy to use as shown in my example.

I'd like to add that you do not completely seem against the idea, considering you didn't even mention the fact that this blocking was an intended behavior. So, is it indeed a bug? And if so, wouldn't it need to be fixed whatsoever?

Hopefully now, with this context, you understand where I'm coming from. Please, let me know what you think! (:

[Joel Christner]

I just checked the locking that is used, and it's set and released in the correct spots. I haven't tested the concept of having a SendAndWait inside of a SyncRequestReceived event. I don't see an immediate reason why this wouldn't work though, based on where and how the SemaphoreSlim objects are used. I'll take a look into that, i.e.

server.Callbacks.SyncRequestReceived += request =>
{
    Console.WriteLine(2);
    var response = server.SendAndWait(1000, request.IpPort, "hello from server!");
    
    Console.WriteLine(4);
    return new SyncResponse(request, response.Data);
};

Is the use case you're looking to implement command/response, i.e. "you give a command, I give you a response", or, is it "you give me a series of commands in sequence, and based on the sequence, I'll respond accordingly"? I guess what I'm asking is, is every command/response interaction discrete and atomic, or, is there history or a decision tree being followed?

If it's the former, there's likely little need to nest these. If it's the latter, I'd definitely implement using a state machine pattern. In that case, I'd recommend drawing up a directed acyclic graph showing the possible interactions and state changes. From there I may be able to help with some scaffolding to define the states, transitions, and provide guidance on how to handle incoming messages.

As far as why I generally don't recommend using SendAndWait - the world is going asynchronous. Yes, the implementation of SendAndWait under the hood is async/event-driven, however, the way you have to build your logic around it is what is difficult, specifically in terms of the number of error conditions you have to handle (which go up exponentially for every level of depth you have in a graph of possible interactions). But, I do understand the convenience of knowing that "this response is for this inquiry".

[Matt]

Hey! Happy thanksgiving!

I honestly don't have much to say here. Yes, the use case I am looking to implement is the former, aka "you give a command, I give you a response".

It would be really great if the given example code could work just fine hahah, but I'm not too worried considering you said this:

I don't see an immediate reason why this wouldn't work though, based on where and how the SemaphoreSlim objects are used. I'll take a look into that, i.e.

I guess I'll wait for you to investigate, as I do not believe there is much more I can do as of right now.

[Joel Christner]

Hi @Laiteux I put together a simple project within the repo (see Test.SyncMessages) which shows how I approach send/wait with command/response types of systems. There isn't really a state machine since every exchange is atomic. Please let me know if this helps.

Commit: 6264a69

Sample output:

C:\Code\Watson\WatsonTcp\src\Test.SyncMessages\bin\Debug\net7.0>test.syncmessages
Client: connected to server
Server: client connected
[Command ?/help]: ?

Menu
----
server send        Send a message from the server
client send        Send a message from the client
server echo        Send an echo request from the server
client echo        Send an echo request from the client
server inc         Send an increment request from the server
client inc         Send an increment request from the client
server dec         Send an decrement request from the server
client dec         Send an decrement request from the client

[Command ?/help]: server send
Data: async from server
[Command ?/help]: Client received message: async from server
client send
Data: async from client
[Command ?/help]: Server received message: async from client
server echo
Data: hello from server
Client received sync request: {"CommandType":0,"Int":0,"Data":"hello from server"}
Response: hello from server
[Command ?/help]: client echo
Data: hello from client
Server received sync request: {"CommandType":0,"Int":0,"Data":"hello from client"}
Response: hello from client
[Command ?/help]: server inc
Data: 5
Client received sync request: {"CommandType":1,"Int":5,"Data":null}
Response: 6
[Command ?/help]: client inc
Data: 7
Server received sync request: {"CommandType":1,"Int":7,"Data":null}
Response: 8
[Command ?/help]: server dec
Data: 9
Client received sync request: {"CommandType":2,"Int":9,"Data":null}
Response: 8
[Command ?/help]: client dec
Data: 7
Server received sync request: {"CommandType":2,"Int":7,"Data":null}
Response: 6
[Command ?/help]:

[Matt]

Hey!

Thanks for getting back to me!

I appreciate the test project, however I do not quite get its point?

I already have a fully working command system, which is the code I showed you. This was real code, not pseudo-code. I have a fully working library already. This not not the problem hahah

All this issue is about is being able to send a SyncRequest from within a SyncRequest, and getting a response.

This really is the only thing that doesn't work. You mentioned you would check the locks and investigate as to why it doesn't work as it should.

Not sure what else to say, I'm a little confused 😅

[Joel Christner]

Yes, understood, I made that to close the loop on how I normally approach sync messaging. I'm still investigating the issue :)

[Joel Christner]

Ok, I stand corrected. I didn't have to pass context to marshal the response. I was able to wrap it in an unawaited Task. Please give v5.0.5 a try and let me know if this solves it for you. It worked on my end :)

NuGet: https://www.nuget.org/packages/WatsonTcp/5.0.5
Commits: 316f880 and 13d8962

C:\Code\Watson\WatsonTcpTest\WatsonTcpTest\bin\Debug\net7.0>watsontcptest
1
2
3
4
client received response: hello from client!

C:\Code\Watson\WatsonTcpTest\WatsonTcpTest\bin\Debug\net7.0>

[Matt]

Hey! Just got back home. Really appreciate your work on this, it's great to see you were pretty easily able to come up with a fix. I'm excited to try it, will let you know very soon (:

[Matt]

Alright, it seems to work with a very simple setup (the code we're testing with), but I am encountering a few issues when trying from "richer" code. I do not believe it is related to this exact issue, but more likely v5 in general. Still, it prevents me from doing useful stuff.

Are MessageReceivedEventArgs.Client / SyncRequest.Client supposed to be null when called on WatsonTcpClient's side? (server->client, just like in the code we're testing with)

If so, shouldn't the property be nullable? And most importantly, then how do you get the server IP:port / Guid from there, just like before (with v4)?

[Joel Christner]

Yes, the Client property is null when receiving an event on the client side, since the client can only have one connection. The connection to the server does not itself have a Guid since there is only one connection. This is a byproduct of changing how client metadata is exposed (from List<string> to List<ClientMetadata> from v4 to v5. Do you have a need for this information in the event on the client side? I assumed no, since this information is needed to instantiate the client.

[Matt]

Hey Joel! Thanks for the quick reply. I hope you had a great weekend (:

Do you have a need for this information in the event on the client side? I assumed no, since this information is needed to instantiate the client.

Well, no, not really. I had some code which was wrongly using it on client's side so I changed it. It's in fact not needed, but I was curious so I thought I'd ask anyway.

[Matt]

Anyway, everything now seems to work just fine. I'll close this issue so we can move to the next one whenever you have some time.

Thanks a lot for your time collaborating on this, it is very much appreciated. Please keep up the good work! (:

[Joel Christner]

Thanks Matt! Will be moving over to the other issues shortly! Cheers