Relax limits on hub discovery
manton opened this issue · comments
Under section 4, the specification lists 3 ways to discover the URL for a hub: HTTP headers, <link> elements in an XML document, and /.well-known. The wording is that "Publishers must implement" one of these 3 options.
I think this could be relaxed to say that "Publishers should implement" one of these. That would open up other ways to specify a hub in other formats. I'm thinking specifically of JSON Feed's hubs field, but this change to WebSub wouldn't rule out a more general way to specify links/relationships in JSON in the future.
It cannot just be a should or you lose general interop. You have to specify some ways for subscribers to be guaranteed to find the websub hub.
Every one of those methods that you add, increases the work for subscribers as well. This is why .well-known is actually being dropped. (as well as its not actually used in practice it seems)
Are http link headers not enough?
XML is allowed for historical reasons and to not break existing implementations, adding anything new will break all existing implementations of clients.
Also xml had namespaces... It's not just <link>. This means that we can avoid collisions easily.
How would it break existing clients? Existing clients that are looking for HTTP headers or <link> should continue to do so.
The reason to not rely only on HTTP headers is the same reason that <link> is supported at all, right? Because you don't always have control over the headers that come back from the server. I thought that including something similar with JSON would let more people use WebSub, as well as promote the idea so that it's more visible. I'd love for WebSub to be even more broadly supported.
It would break interop with compliant publishers that use the new method. Basically, adding that makes fully compliant clients no longer be fully compliant, as they cannot detect hub links from all fully compliant publishers.
The spec requires that you implement at least one of these options. It doesn't forbid implementing others, including experimental ones. So whatever you specify for hub discovery from JSON Feed documents is fine, for those clients that understand it. This is true whether the word in the spec is "must" or "should".
Hmm. Looking at it from the discoverer's point of view, though, the current spec requires that discovery first try HTTP headers, then XML link elements if appropriate, then host-meta; but is silent on other potential mechanisms.
Maybe it'd be better to say that you have to try all N standardised mechanisms, but information you learn from HTTP headers overrides that from link elements, which in turn override information from host-meta.
That way, you specify the necessary parts of a partial order of preference, without forbidding new discovery techniques from being slotted in as appropriate.
For example, at the moment, one reading of the spec forbids use of JSON Feed discovery from being treated as preferable to host-meta discovery.
@tonyg @manton As it's the feed implementer, not the spec, that needs to implement one of the existing discovery mechanisms, any additional discovery mechanisms would be plain redundant in the current case.
@manton Current libraries that support fetching updates through WebSub would need to have their discovery mechanism extended and/or made pluggable to support the new property or else would not detect that the feeds are realtime and then either ignore the feed or fall back to polling.
So anyone currently fetching WebSub feeds with a compliant library would have to extend their WebSub library to properly detect all JSON Feeds, or else fail to detect the realtimeness.
I agree that it would be neat to support JSON feeds without having to rely on headers, I would otherwise be unable to support WebSub for any JSON feeds in my Jekyll site. (And same is on the other hand true for any other non-XML/HTML WebSub updates. Like updates of a PNG, to take an insane example. Only solution for a static site to allow WebSub updates for any data format would be to fix the host-meta discovery 😛)
Perhaps one disconnect here is that we're talking about the risk of existing libraries no longer discovering the hub, but those libraries won't work with a new format anyway, unless the feed parsing is completely isolated from the WebSub support. Possible, but strikes me as unlikely.
Two other points while I'm here... :-)
- I agree that host-meta should be deprecated. I'll be surprised if it's used much in practice.
- I noticed that WordPress.com, which must be a very popular source of WebSub links, only uses
<link>and not HTTP headers. For whatever that's worth.
I think we can close this issue. Re-reading the comments, I'm not seeing a conflict between JSON Feed and the WebSub spec, other than that HTTP headers should be preferred and always checked by a client. If I'm wrong, please let me know. Thanks!
So we agree that the JSON Feed discovery mechanism works with the current wording of WebSub and that neither has to be changed?
JSON Feed publishing a hub property is not supported by the spec (especially since it doesn't publish a 'self' link either). This is actually the same issue with host-meta, its somewhat pointlesss as there is not current way to publish a self link as well without extending existing methods.
To the ways forward you bring up...
- JSON Feed switches to one of the WebSub suggested discovery mechanisms
this would mean publishing link headers
- WebSub makes implementing one of the specced discovery mechanism an optional part of the spec
this would be bad for interop as making discovery methods optional means there will be no guarantee of being able to subscribe.
- JSON Feed decides to not be WebSub compliant
being websub compliant or not is not within JSON Feed's control, its outside of the spec and entirely separate from it. The JSON spec cannot break support for HTTP.
(4. The current XML and HTML specific solutions gets extended to all formats that supports some kind of web linking / link relations and JSON Feed decides to implement such web linking: manton/JSONFeed#44)
The only reason XML / HTML is there is essentially for backward compatibility with how it was done in the past. If you start to support more formats, you will have to start making some for every format that comes along. There will be tons of discovery mechanisms and this will put undue burden on subscribers.
Just a quick point, @dissolve: there's no self but JSON Feed has feed_url which is exactly the same thing. In the JSON Feed spec it says to use feed_url for hub.topic.
Yes, I noticed that, I think that would be even worse if every format had its own naming of the links. I can see some mythical format EDIFeed (that would be horrific) using feed_url to describe the HTML version of the feed and not the url of itself.
We should keep this issue closed and start over. If someone starts a new GitHub issue, feel free to mention me on it, and I'll try to provide my 2 cents from the JSON Feed side.