palewire / campaign_finance

Campaign Finance Dashboard

This project was built at CALmatters to make California's public data about campaign finance easier to analyze and understand. The data was scraped from Cal-Access, the state's messy government database about lobbying activity and politics.

Getting Started

The scraper was built in Python 2.7 using the web crawling framework Scrapy. The dashboard was built using Mongo, Express, React, and Node.

0. Prerequisites

• MongoDB 3.6+
• pip
• npm
• yarn

1. The Scraper

To get started, cd calaccess_scraper and run pip install -r requirements.txt to install all dependencies. The scraper will require a MongoDB URI in a secrets.txt file, which is not included in the repository. Place your secrets.txt file in /campaign_finance/calaccess_scraper. In the same directory, run scrapy crawl [spider name]. The four spiders currently available are:

• ballot_measures
• funds
• committees
• candidates

2. The Dashboard

Install all dependencies by running yarn install in the root directory of the project. Afterwards, run cd backend && touch secrets.js. Store your MongoDB URI in your secrets.js in the following format:

const secrets = {
  dbUri: "your-uri-here"
};

export const getSecret = key => secrets[key];

Run the full app locally with the command yarn start:dev. You can also run the client or the backend separately with the commands yarn start:client and yarn start:server.