Failed to fully download large group
dado3212 opened this issue · comments
I'm a member of a very large group (>14k topics), and I tried to use your script to fully download all of it. However, I only extracted ~4k entries into the generated 'mbox' folder, and is missing a lot of the older messages. Any idea on why your script wouldn't get all of them?
Whoops, didn't realize that ./crawler -sh had to be run through completely first.
Awesome, @dado3212 !!
I am also having this issue with a large group. The crawler seemed to stop in the middle for no reason.