KeithCu / LinuxReport

Customizable Linux news site based on Python / Flask

Home Page:https://covidreport.keithcu.com/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Prevent excess RSS requests

KeithCu opened this issue · comments

When a feed has expired, if two people come to the website at the same time, it will possibly fetch for both of them. Implement some logic which if the feed has expired:
Check for URL + "FETCH" in the cache
if it doesn't exist, then create an entry containing this PID / TID
Then check to make sure it's our PID / TID

If it already exists and it's not our PID / TID, sleep for 100 ms and keep checking till cache entry disappears
If it is, then fetch the RSS feed, add to cache, and then delete the URL + FETCH cache entry