Getting continuation while ordering by oldest first repeats first article of continuation

[Daniel Cachapa]

We're getting 30 items at a time, and using continuation to get another batch of articles when the user reaches the end of the list.

When the order is newest first it works fine, but when we request oldest first (r=0), the first article of the list retrieved through continuation is a repeat of the last one from the previous list.

In testing, we managed to get around this by casting the continuation id to long and incrementing it before submitting, but this is a hack we'd rather not have to rely on.

Here's a sample request for Gizmodo:

GET https://theoldreader.com/reader/api/0/stream/contents?s=feed%2F530f708bc70bc262a2000512&xt=user%2F-%2Fstate%2Fcom.google%2Fread&r=o&c=&n=30&output=json

[Dmitry Krasnoukhov]

Thanks for reporting. Let me do some testing and I'll get back to you shortly.

[Dmitry Krasnoukhov]

Hey @cachapa, unfortunately I had no luck with reproducing of this bug.

First thing I've added API spec to our internal testing system that checks how both reverse sorting and continuation works, and it showed that everything is working fine.

Then I've tried to reproduce it with subscription ID that you mentioned, no luck again. However, it looks like you've marked some of its items as read, so I checked it w/o xt parameter as well, same results.

Regarding the issue itself: I assume that duplicated item causes confusion, but it probably should not, since there is a chance that you'll retrieve the same item with any other request. You can use unique item identifier to ensure items uniqueness in your local storage. I hope that this can help you.

Let me know what you think.

[Daniel Cachapa]

Thanks for taking your time to check on this.

I'm still able to replicate this consistently, so I'm surprised you couldn't do it from your side.
The conditions seem to be very specific to trigger the bug: when we're requesting only unread articles (xt) and ordered by oldest-first (r=o).
Additionally, we're requesting 30 articles each time, though I've tried with other values and the problem occurs as well.

Here are two requests which for us always show the problem:

First list:

https://theoldreader.com/reader/api/0/stream/contents?s=feed%2F531c92e4c70bc2e172000392&xt=user%2F-%2Fstate%2Fcom.google%2Fread&r=o&c=&n=30&output=json

Continuation:

https://theoldreader.com/reader/api/0/stream/contents?s=feed%2F531c92e4c70bc2e172000392&xt=user%2F-%2Fstate%2Fcom.google%2Fread&r=o&c=1395630114&n=30&output=json

For now, we've been able to get around this by doing exactly what you suggested: we check the id of the "border" articles and if they match, we remove the duplicate before joining the lists.

Can you expand on the chance that we'll receive the same article in other requests? I'd prefer not doing these checks at all if possible, as they seem to be a bit hacky to me. We've not seen this issue in our implementation of other APIs.

[Dmitry Krasnoukhov]

Thanks for detailed report. You're right, this issue was pretty rare and it was possible to reproduce only with exact set of given parameters. It's now fixed so you can get rid from your hack.

I assumed that you're using local cache/storage to sync with our API. So, there might be case when you retrieved some items from subscriptions, and the same items are in other feeds like starred, etc. I'm not sure it's relevant, but it makes sense to cache/store items with unique ids anyways.

[Daniel Cachapa]

I confirm this is fixed.
Thanks again for the quick turnaround.