Clarify whether fat pings are required or not
voxpelli opened this issue · comments
Following the discussion in #27 it's apparent that the current language in section 6 isn't clear enough on whether fat pings are required or not and we should clarify that to avoid confusions going forward:
A content distribution request is an HTTP [RFC7231] POST request from hub to the subscriber's callback URL with the payload of the notification. This request must have a Content-Type corresponding to the type of the topic. The hub may reduce the payload to a diff between two consecutive versions if its format allows it.
It should be clarified what the payload sent with the notification is actually expected to contain and whether an empty payload is allowed. Some SHOULD, MAY, MUST language regarding the payload and the corresponding Content-Type would be good to ensure that subscribers and hubs knows what they can expect from each other and how to handle the different cases.
Pinging @julien51 and @aaronpk here as I believe you both have something to say in this regard.
In today's call, we discussed that hubs should be required to send fat pings, at the very least sending the full document without doing any diffing mechanism. If we can document the current content types that have well-defined diffing mechanisms (#27) then we can recommend hubs do that. But at the very least, subscribers should expect to receive the full document in the notification payload. Of course it's up to the subscriber whether they actually do anything with that content, since the subscriber is always free to fetch the document from the original URL if they want.
I wanted to differentiate the "fat" from the "diff" issue here. The former is relatively simple. The latter, not so much.
Defaulting to the "full" content is good and may be the only thing that's "spec-able". Yet, I want to explore if there is any chance we can provide diffing in a predictable way.
I think a good default is for hubs to send the full content. As a subscriber of an arbitrary PubSub-enabled feed, I would certainly not be surprised to received the full document as the payload. I would, however, be surprised if the payload was a diff, unless it was clearly documented in the spec what that would look like. If I as a subscriber received a payload I didn't recognize because I wasn't anticipating the diff'd payload, I would just go fetch the topic URL myself, defeating the purpose of the fat ping.
I agree it's challenging to describe an "arbitrary" diffing mechanism for any content type. However we don't actually have to do that. All we have to do is describe what diffing mechanisms implementations have actually done and for which content types, and codify that in the spec. I'm guessing we can make a reasonable assertion of the best way to deliver diff'd notifications for RSS and Atom feeds, and we can probably also do the same for JSON Activity Streams 2.0.
@voxpelli does this clear it up for you?
@aaronpk Yes!
for future reference
I think there's a reasonable technique for adding diffs as an extension, if and when people want.
Basically: when you get the a fat-ping notification, if you'd rather get patches in the future, include an Accept-Patch header ( https://tools.ietf.org/html/rfc5789#page-7 ) in your response. If the hub can do diffs using one of the media types you named, then future fat-pings will be done using the PATCH verb with that media type.
This suggests the fat-ping notifications would ideally be done using PUT instead of POST, but that's the kind of change that's not really warranted at this point in deployment.
Probably one could also send the Accept-Patch on the subscription confirmation, to avoid ever being sent the full content.
I'd want some careful thinking about etags, so that could know if you missed a patch. I haven't looked at how one might do that. Possibly it can be done with existing if-match, etc, headers.