Jackustc / Crowdfunding

Scraper for obtaining crowdfunding data in a structured manner implemented in Python with Scrapy.

Crowdfunding-scrape

Scraper for obtaining crowdfunding data in a structured manner on fundrazr.com implemented in Python with Scrapy.

Usage

To crawl the scraper, you need to install Python 3, as well as the Scrapy framework.

To install scrapy type the below command in the terminal:

  pip3 install scrapy  

To crawl the main spider for scraping pages, first navigate to the project folder then simply run the command

  scrapy crawl my_scraper

will scrape all information from https://fundrazr.com/find?category=Health. By default, the scraped data will be stored (using Scrapy's feed export) in the Data directory as a (.csv) file following the naming convention:

{Current UTC time}.csv

If you prefer a different output file name and format, you can specify this from the command line using Scrapy's -o option. For example,

  scrapy crawl my_scraper -o Source1.csv

will output the data in CSV format as Source1.csv. (Scrapy automatically picks up the file format from the specified file extension).