Recommended API to set socket options on ZMQ?
acowlikeobject opened this issue · comments
I've been playing with inproc and ipc (using ZMQ), and have encountered a problem where my messages are dropped unless I put a sleep after each publish.
There are several threads online that talk about this problem (e.g., here and here). Most suggestions require access to the ZMQ socket or the context, so I was wondering, what is the recommended way to specify ZMQ socket options (e.g., zmq_setsockopt(socket, ZMQ_LINGER, 0, sizeof(int))) or to set them after zcm_create since they're created behind the scenes by zcm?
I don't think I'm following the issue well enough. Can you explain your use case? Is it that your first message doesn't actually get published? If so, that is a known issue with ZMQ that I have no idea how to fix. If you know how, please enlighten me haha. If you're simply saying that your message isn't publishing because your program closes too early, we can certainly look into fixing that for you. If this is the case, have you tried calling zcm->flush() in the cpp api or zcm_flush(...) in the c api?
I don't think I'm following the issue well enough. Can you explain your use case? Is it that your first message doesn't actually get published? If so, that is a known issue with ZMQ that I have no idea how to fix. If you know how, please enlighten me haha. If you're simply saying that your message isn't publishing because your program closes too early, we can certainly look into fixing that for you. If this is the case, have you tried calling zcm->flush() in the cpp api or zcm_flush(...) in the c api?
Initially I thought messages were being dropped because the buffer was not flushed before shutdown. I don't think this is the problem (although zcm_flush() looks very handy, thank you!).
Turned out there were two other problems. First, as you say, the first message doesn't get published. I don't have a solution but am going to try something clumsy like sending dummy messages as a handshake.
Second, in the publish.c example, if I reduce the sleep to < usleep(30) (from usleep(1000000)), I start dropping messages (return code -2). I imagine I'm overwhelming the socket buffer at that point, so I wanted to experiment with realistic send frequencies and the HWM / block vs. drop settings on the socket.
My two questions are (both about accessing underlying zmq objects, I guess):
- Per the subject, is there a way to
zmq_setsockopt(to play with HWM, etc.)? - Rather than using
usleepto throttle the send, which is unreliable / fragile, is there a way to poll the socket(s) from withinzcmto know which are ready to send?
Can you describe your use case here a bit more? Zcm is intentionally designed to be transport agnostic. So needing access to the underlying mechanisms is exceedingly rare. Is there a reason you care if messages are dropped? Zcm is designed to be a pub/sub framework with absolutely no guarantee of message delivery (think of it like UDP).
Yep, figured there are no delivery guarantees in zcm. But exactly how/when messages are dropped can be affected by the zmq configuration, and I thought it natural to want to tweak that to better fit our use case (which is outside of robotics, where certain channels need to have stronger guarantees).
But I respect the zcm design choices (especially since I'm applying zcm to a different domain than it was built for), and if there isn't a way to access the underlying transport layer, that answers my question and I can close the issue.
You can always wrap the zmq transport yourself and expose whatever functionality you'd like. Then you can just instantiate the transport and create a zcm instance through a call to zcm_init_from_trans(...). May I ask what use case this is for? Might be an even easier way to do what you need if you're able to disclose
Unfortunately, I can't get too much into the specifics of the use case here, but basically, I know what our max. throughput will be on each channel, and I'd like to block once we hit that. I'll use zcm_init_from_trans() as you describe to set high water mark / block (instead of drop) behavior / send buffer size on the socket to accommodate that throughput and see if that behaves more in line with what we need.