lose package or wrong order
flier268 opened this issue · comments
I have two process, A send package, B receive package.
I use RemoteRequestWithoutResponse function to send package.
(a package is 389 byte)
The issue is, it lost some package or wrong order sometimes.
(both of 1.16 and 1.17 )
I don't think so it is a production system, probably you do smth. wrong..publish your test solution with detailed comment
When you call RemoteRequestWithoutResponse check returned boolean value. If it is not true, then your package can be out of sending queue and there should be an exception logged.
The wrong order is not possible by design if you send from the same thread. So, please check the code and try to send me the project that emulates that behavior.
I didn't see any packet loss for now, but concerning wrong order: data comes into Process2 in correct order, but receiving that data thread should not be blocked by your processing/callback function. That's why your callback is called from Task.Run
Task.Run makes these mixtures. Without Task.Run you will not receive messages until your callback finishes processing. The problem of logical sequencing of messages is already a part of higher level protocols. You must implement it somewhere here:
You can't rely on msgId, because Process1 generates msgId, within Process1 lifecycle the Process2 can be restarted several times and msgId can be quite big (from the point of view of the new born Process2). That's why only higher level protocols encoded in (byte[])data can do sequencing job.
You can make a tweak inside of SharmIPC. Call AsyncRemoteHandler without Task.Run.
Then parse your package and call Task.Run (()=>ProcessAParsedPackage()) inside of your callback function.
Test different way, then, may be, we can integrate activation of such behavior via parameter of SharmIPC initializaiton.
Maybe we can add a buffer on Process 2
Then let process 2 calc it starting from first one msgid received
Then waiting for next msgid,if received worng order id, add to buffer and wait for next one.
If received correct one, call event and post all of data from buffer
There is no understanding of correct msgId, because process2 can be restarted many times, when Process1 goes on to live and generate ID. MsgId is internal messaging counter. You must take care your own ID cycle encoded in data. We mostly (95 % of time) call single data transfer. Sometimes we supply huge data package (more than 50 MB), in this case we split it into chunks, create collecting and reordering buffer. This architecture is ideal for that.
Next thing, if you remark Task.Run you will see the correct sequence. I recommend you to have own sharmipc variant with remarked Task.Run. In your callback parse( message) and then run Task.Run(()=>process(message)). Take a look may be that will fit your needs.
I had fix it.
SharmIPC.zip
Please find out Process1\SharmIpc\SharmIpc.cs and compare it with git
Any other better idea?
I see, I guess that git version will not be changed very often, so you can work with your version and check the long term effects.
For my scenario, I must say that Process1 generates messages for Process2 where they have to be processed in different Threads and it means that after the Thread in SharmIpc constructor (that you made) another one Thread should be spawned - overdose of threads. I still prefer message reordering after AsyncRemoteCallHandler is called.
Maybe we can add an option to switch it.
Default config is your version
What do you thing?
I can make smth. like this, please confirm:
I have uploaded sources to git, grab and test:
But, if AsyncRemoteCallHandler on Process 2 is slow(have some Thread.Sleep(200);), Process 1 will be lock
Of course, but you are free to implement your own solution inside of AsyncRemoteCallHandler, like you made it in example with extra processing thread.
Sound good!
By the way, it need to be response,not RemoteRequestWithoutResponse
I have a confused about its name
Very contradictory, right?
Ok, then I will publish new version in NuGet.
From docu: RemoteRequestWithoutResponse "just sends payload to remote partner without awaiting response from it", so I didn't get what you mean.
Though, sometimes to choose the correct name for the function is harder than writing the function itself.
Released v1.18