pchr8 / up_crawler

Script that downloads articles from Ukrainska Pravda

pchr8/up_crawler Issues

Fix broken download of older articles
Updated 4 months ago
Remove newline in UP Crawler polite header
Closed 5 months ago
Fix bug with sitemap in the future
Closed 5 months ago
Improve tags_map writing and interruptions logic by writing to a temp file and then moving to real location
Updated 6 months ago
Allow filtering downloaded articles by language
Updated 7 months ago
Allow downloading more recent UP articles through news sitemap xml
Updated 7 months ago
Set up pypi / distribution / versions etc.
Updated 7 months ago