Reddit rss feeds stop working after last update

Question

Reddit rss feeds stop working after last update

samcro1967 opened this issue 2 years ago · comments

samcro1967 commented 2 years ago

Seems like the update a few days ago effected Reddit RSS links.

Reddit Link

nkanaev · Answer 1 · Mon May 09 2022 07:15:31 GMT+0800 (China Standard Time)

this is very weird.

It seems that either rate-limiting or request blocking is at play. The former happens when there are too many requests to the server. I've seen the latter when "user-agent" header is not present.

In my case it's 403 Blocked. It is possible Reddit is experiencing internal issues (or blocked the application 🤷 ).

samcro1967 · Answer 2 · Mon May 09 2022 09:05:58 GMT+0800 (China Standard Time)

I spun up Freshrss to to see if it exhibited similiar behavior and it is working. Tried to readd to Yarr and it is still failing. From what I can tell, it seems like Reddit is working fine.

nkanaev · Answer 3 · Tue May 10 2022 21:37:16 GMT+0800 (China Standard Time)

My findings so far:

With the current setup it's always returning 403 Forbidden
The issue is not related to the latest release. I've checked it on v2.2, and still getting blocked.
Using http.DefaultClient instead of a custom one in src/worker/client.go results in 429 Too Many Requests.

This is what the go client sends in HTTP headers (checked locally):

GET / HTTP/1.1
Host: localhost:7071
User-Agent: Yarr/1.0
Accept-Encoding: gzip

Trying to reproduce it with the headers via Curl doesn't result in 4XX status code. Running:

curl --http1.1 -H 'User-Agent: Yarr/1.0' -H "Accept-Encoding: gzip" https://www.reddit.com/r/selfhosted.rss

works fine.

I'm pretty sure the issue is caused by Reddit. There's the same error discussion in miniflux/v2#1432.

Upd. relevant discussion in https://www.reddit.com/r/redditdev/comments/ukr3ld/miniflux_rss_reader_stopped_working_yesterday/

nkanaev · Answer 4 · Thu May 12 2022 16:47:57 GMT+0800 (China Standard Time)

Seems to be working now.

mokahless · Answer 5 · Wed Jan 04 2023 01:46:07 GMT+0800 (China Standard Time)

I just want to note for the record that I had this exact issue last happen in November. If it is going to re-occur, we at least need a workaround. Sorry, I've been deleting and re-commenting. I hope this is the last time and the most appropriate thread for me to post in.

Linking #1472 #1628 #1432 as related.

I think we need some kind of checkbox to force certain feeds to not stop checking after an error, or perhaps have them on a timeout we can configure, rather than just have them permanently give up.