Topic "." workaround should work in both directions
joe-average-user opened this issue · comments
Currently dots in topics are replaced by "/" when publishing, like:
owntracks/user@domain/tld/samsung s7
which leads to problems when subscribing is:
owntracks/+/+
It would be a lot better to make this "." to "/" patch also work in the opposite direction, meaning publish should look like:
owntracks/user@domain.tld/samsung s7
Like it would on all other mqtt implementations.
You made this patch only in the incoming direction from publisher to amq.topic exchange, but not in the outgoing direction from amq.topic exchange to subscriber. An mqtt subscriber is not an amq subscriber and should therefore receive published messages that are correct for his standard.
This is pretty much a duplicate of #58.
You are talking about cross-protocol interop, so the argument about "all other MQTT implementations" does not apply. The only known problem for MQTT-to-MQTT communication is #58.
Replacements in topics are not always safe and cross-protocol interoperation often brings up edge cases that no protocol spec dictates how to handle. What if an exchange is used to publish to queues used by both AMQP 0-9-1/1.0 and MQTT consumers?
We are interested in finding a way to avoid some of these issues entirely but currently
NAK, I am not talking about cross-protocol usage. I tried to explain how the mqtt publishes walk through your software from the mqtt-plugin over to amq part (where your character replacement obviously works), but back from the amq exchange to the subscribed MQTT listener it does not work as expected, because the character replacement in the other direction is not there.
For the pure MQTT user there is no amq protocol and it should - if there - be completely transparent. But now it is not. A MQTT subscriber has to subscribe to 3 levels in my example on every other mqtt server, but on RMQ he has to subscribe to 4, because the dot in his full username (including the domain part) adds an additional level in your amq software part.
@joe-average-user we'd appreciate an executable way to reproduce instead of explanations how "our software" works. Dots and slashes in topic names is a known limitation mentioned in the docs and what #58 already covers.
It would be nice if "our software" didn't have that limitation but it happens to provide cross-protocol interop and this is where the topic/routing key translation comes from. If you have specific ideas about how #58 can be addressed in a safe way, please leave a comment there.
It is not nice to tell people addressing an obvious problem (and in fact it is a big one) to go and know your code first. You are clearly violating the given mqtt standard, here:
http://docs.oasis-open.org/mqtt/mqtt/v3.1.1/os/mqtt-v3.1.1-os.html#_Toc398718109
It says a topic is an "UTF-8 encoded string". There is no reference that a dot "." should not be used.
If you want to do a working mqtt-plugin there should be ways to work around the immanent problems of your engine with certain characters without letting the user know about it. I am pretty sure that reversing your dot-to-slash patch "on the way out" to the subscriber is possible, be it by conversion callback into the plugin or else. But to tell you exactly how needs an indepth review of your code - for me. But not for you, as you already know it perfectly well.
@joe-average-user the section linked does not say that implementations are not allowed to limit certain characters from use.
Our research into #58 suggests that there are edge cases in topic (routing key) segment translation that are not as trivial as some make it sound. We welcome pull requests or a description of a specific algorithm and test cases that would improve the usability of this plugin.
We do not welcome patronizing comments that ignore why things work the way they do, and that the limitation has been long documented.
In case it is not obvious why "replacing on the way back" is not a no-brainer: what if the topic had slashes that were not meant to be separators? Other protocol specs do not prohibit that. They would be replaced with dots and voilà, a different set of cases and users will have the same problem which they did not have before. Which is what #58 originally was about, a way to escape separator characters.
A workaround without addressing this fundamental issue that stems from protocol interoperability supported by RabbitMQ would lead to unintentional character replacement in topics.
FWIW this issue is brought up about once a year: not so big after all.