Removing messages from zmqpp::message object
sunkin351 opened this issue · comments
Can someone tell me if there is a way to do this? Could someone implement this if there is no way?
Even a simple zmqpp::message::clear() member would be great.
There is a void message::remove(size_t const part) and both void message::pop_back() and void message::pop_front() to remove parts of a message. If you need to remove the whole one it's probably faster to use either swap mechanics or the move assignment operator with an empty message.
Can I ask what your use case for this is? Messages are 'cleared' automatically when sent, because zmq takes ownership of the memory, or when receive is called on a message with data (https://github.com/zeromq/zmqpp/blob/develop/src/zmqpp/socket.cpp#L211)
I guess for what I want to be efficient, we would have to change the message container from a vector to a deque, which is more efficient than vectors in manipulation of the front elements of the container (Not to mention it has all the main features of a vector such as referencing elements by number). I want to delete the extracted parts from message objects as the parts are being read, so that, for one, I could accurately tell how many more messages that are left, and two, save some memory when reading big messages. (Also, pop_front() on a vector isn't really efficient at all.)
May I also ask for a better system for message part type detection? Currently, all it does is assert when you try to extract an int when you really have a long int in the message... Maybe use exceptions instead? I would really like my program to have the possibility of being able to recover when encountering an error like this.
My main issue with deque is that, as far as I'm aware, most implementations tend to be fairly inefficient for small lists. I'm not sure about you but certainly I tend to have hundreds of thousands of message a second and each with fewer than 3 or 4 parts. That said there is likely no reason we can't make zmqpp::message a template type that can happily use either and just default to vector.
With regards to the asserts, those get compiled out by any release build anyway so don't act as any protection outside a debug build (anything with NDEBUG defined will remove assert), and we are only tests for size of elements anyway not type so we have no guaranty that those 4 bytes you are reading are really an int. I'd be happy to add some 'safe' versions that did this check and threw exceptions but I wouldn't want that overhead normally when it's not really checking against anything other than part size.
May I also ask for a better system for message part type detection?
This is impossible. The only thing you can hope for is that the size match.
You cannot distinguished a 4bytes non-null terminated string from
uint32_t for example.
They both have a size of 4 bytes.
I could accurately tell how many more messages that are left
You can actually. You can call message::remaining().
I'm not convinced that replacing vector with list/dequeue will yield improvement
in the general use case. However if it proves useful in some special case,
I like Ben's idea of allowing zmqpp::message to be templated on the storage
type.
On Fri, May 20, 2016 at 10:13 AM, Ben Gray notifications@github.com wrote:
My main issue with deque is that, as far as I'm aware, most
implementations tend to be fairly inefficient for small lists. I'm not sure
about you but certainly I tend to have hundreds of thousands of message a
second and each with fewer than 3 or 4 parts. That said there is likely no
reason we can't make zmqpp::message a template type that can happily use
either and just default to vector.With regards to the asserts, those get compiled out by any release build
anyway so don't act as any protection outside a debug build (anything with
NDEBUG defined will remove assert), and we are only tests for size of
elements anyway not type so we have no guaranty that those 4 bytes you are
reading are really an int. I'd be happy to add some 'safe' versions that
did this check and threw exceptions but I wouldn't want that overhead
normally when it's not really checking against anything other than part
size.—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
#162 (comment)
Kapp Arnaud - Xaqq
Is there a way to create a template class with default parameters without having to use classname<> instanceName; instead of just classname instanceName;? I feel this might be a little bit of an api breakage.
I believe the trick here is to use typedef or using.
Create a zmqpp::message_impl<T> class that is templated on the storage type.
Then, provide a typedef for zmqpp::message by declaring using zmqpp::message = zmqpp::message_impl<std::vector> for example.
For your particular case, in your application code, declare using sunkin::message = zmqpp::message_impl<std::list>.
Internally, we would have to replace all functions that take a zmqpp::message so that they now take a zmqpp::message_impl<T>. It sounds more involved than I initially thought, but I think this is the best way to go about this. Thought I might be wrong, so idea are welcome !
The unfortunate side-effect of making a template class is that all member functions have to be templates too. I am leaving much of the code unchanged, and adding a static_assert to the constructor to make sure it is only using an operator[] compliant container, Which is only vector and deque as far as I know. I was just wanting deque support for its ability to natively add to/remove from the front of the container. If you would like to also support some of the other containers, I am not sure what that would look like.
UPDATE: I have made most of the necessary changes to message.hpp/.cpp but need help in tracking the rest of the changes needing to be made to accommodate a template class. One place that needs checking would be the zmqpp::socket class.
UPDATE2: Now that I look at this, it might be necessary to have member function templates everywhere there is interaction between zmqpp::message and zmqpp::socket. Are you guys sure you want to go through with this? This is very heavy reliance on templates and will very much increase compile time.
Have I ever told you guys at zmqpp headquarters how much I hate how long it takes for you to reply?
Um... Not to be rude, but have you people dropped off of the face of the planet or something? Where are you? Could you at least determine whether to accept my PR or not?