Redis-based components for scrapy that allows distributed crawling. Small update to make it work for Scrapy 0.16+ and added QUEUE_TYPE and DUPE_FILTER options
Geek Repo:Geek Repo
Github PK Tool:Github PK Tool