Non Zero-Copy Implementation in ZPull
ymjiang opened this issue · comments
Lines 591 to 598 in f50108b
While ZPull
claims to be zero-copy, the PullImpl
that ZPull
calls actually involves memcpy, which can hurt performance. Is it possible to remove them?
@ymjiang did you figure it out?
@eric-haibin-lin Hi, finally I realize this is impossible if we keep using ZeroMQ. Because when receiving a message you need to first declare a ZMQ buffer (whose address is random and cannot be controlled), so on receiving you will have to copy the data from the ZMQ buffer to your desired address.
However, if you use RDMA rather than ZMQ, there is a chance to do in-place pull without copy, i.e., a real zero-copy pull. We are planning to release the code soon.
Look forward to the RDMA PR :)
@eric-haibin-lin Hi, we just open source our RDMA patch. See #124 (we just leave a new comment) for details.