evancofer / reddigest-scalafied

A rewrite and revamp of reddigest

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Determine last link seen timeout period for fetching links from reddit.

evancofer opened this issue · comments

Reddit's JSON api will only allow us to fetch 100 links at a time (https://www.reddit.com/dev/api). This presents a problem, as if all 100 of those links have been seen before, we need to use a link id for after to find posts after the last of these 100 posts. After that, each iteration of 100 posts should do the same thing. However, this is rather time sensitive, as after is not chronological, but instead correlated to ranking. We will likely need to store it in the cache of the user, and then timestamp their last access to reddigest in the database. With this, we can see if the users after post is too old or not. On the other hand, if the after post is still new, it's not too much of a problem, as the user can then just fetch and change after until they find new posts.