nsqio / nsq

A realtime distributed messaging platform

Home Page:https://nsq.io

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

a possible message lost?

liormessinger opened this issue · comments

Hi, I'd like to point out a possible message loss, despite the guarantee stated in the documentation. We have found a way to prevent it but i wondered if there are others. Here's the scenario

  1. time 0:00: consumer on channel A subscribes via nsqlookup, start polling for the topic T
  2. time 0:00: topic T is not found, 404
  3. time 0:01s: producer message sent to topic, topic created
  4. time 0:02s: consumer on channel B looks for topic, receives message
  5. time 0:02s: message deleted (nsq thinks there no more consumers)
  6. time 0:15s: channel A looks for topic, subscribes (sends RDY 1)

which means, for first document, first message for consumer A will not arrive!

We tried to send a warmup message after each subscription, but that's a problem. not always we have a way to have a producer.
we tried to use /topic/create post request, to create the topic and channel ahead of time - but it doesnt seem to work

any other ideas?

many thanks
Lior

Answered in google group mailing list. tl;dr - pre-creating channels should work. You can also pre-create them in nsqlookupd and via nsqadmin (to nsqlookupd). nsqd should look up the set of channels for a topic when creating the topic locally, specifically to avoid the problem of a message being missed on some channels.