istresearch / scrapy-cluster

This Scrapy project uses Redis and Kafka to create a distributed on demand scraping cluster.

Home Page:http://scrapy-cluster.readthedocs.io/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

istresearch/scrapy-cluster Stargazers