mateus-n00b / g1-web-crawler

g1-comments-reader

@author: Mateus Sousa

Usage

• First, you'll need to define your desired G1 URL at the line 13 (g1_crawler/g1_crawler/spiders/g1_spider.py)
• Second, go to the dir g1-web-crawler/ and run scrapy runspider g1_crawler/g1_crawler/spiders/g1_spider.py
• A directory with the output files is created.