In this repo I place some interesting crawler programs developed with Scrapy. These programs are built for enriching the knowledge base on different purpose.
Pass the following argument to terminate the crawler once the number of pages is reached.
-s CLOSESPIDER_PAGECOUNT=1