Non Zero-Copy Implementation in ZPull

[Yimin Jiang]

ps-lite/include/ps/kv_app.h

Lines 591 to 598 in f50108b

 for (const auto& s : kvs) { 
   memcpy(p_vals, s.vals.data(), s.vals.size() * sizeof(Val)); 
   p_vals += s.vals.size(); 
   if (p_lens) { 
     memcpy(p_lens, s.lens.data(), s.lens.size() * sizeof(int)); 
     p_lens += s.lens.size(); 
   } 
 } 

While ZPull claims to be zero-copy, the PullImpl that ZPull calls actually involves memcpy, which can hurt performance. Is it possible to remove them?

@mli @CodingCat

[Haibin Lin]

@ymjiang did you figure it out?

[Yimin Jiang]

@eric-haibin-lin Hi, finally I realize this is impossible if we keep using ZeroMQ. Because when receiving a message you need to first declare a ZMQ buffer (whose address is random and cannot be controlled), so on receiving you will have to copy the data from the ZMQ buffer to your desired address.

However, if you use RDMA rather than ZMQ, there is a chance to do in-place pull without copy, i.e., a real zero-copy pull. We are planning to release the code soon.

[Haibin Lin]

Look forward to the RDMA PR :)

[Yimin Jiang]

@eric-haibin-lin Hi, we just open source our RDMA patch. See #124 (we just leave a new comment) for details.