This is the source code behind our project to collect political ads on Facebook. Built versions are available for Firefox and Chrome. You can browse the American ads we've collected at ProPublica, and the Australian ads over on the Guardian's website.
We're asking our readers to use this extension when they are browsing Facebook. While they are on Facebook a background script runs to collect ads they see. The extension shows those ads to users and asks them to decide whether or not a particular ad is political. Serverside, we use those ratings to train a naive bayes classifier that then automatically rates the other ads we've collected. The extension also asks the server for the most recent ads that the classifier thinks are political so that users can see political ads they haven't seen. We're careful to protect our user's privacy by not sending identifying information to our backend server.
We're open sourcing this project because we'd love your help. Collecting these ads is challenging, and the more eyes on the problem the better.
- Download the Facebook Political Ad Collector for Firefox
- Download the Facebook Political Ad Collector for Chrome
The extension popup is a preact application and you can build a development version by running the following:
$ cd extension
$ npm install
$ npm run watch
If you are a Firefox user you can open a clean browser instance with:
$ npm run ff
and any changes will automatically refresh the extension. (You'll need webpack installed globally.)
In Chrome you'll need to add an unpacked extension by following these directions.
First, make sure you have Rust installed:
$ curl -sSf https://static.rust-lang.org/rustup.sh | sh
Be sure to add ~/.cargo/bin (or wherever cargo is installed, path/to/.cargo/bin) to your PATH. You can do this by adding this line to your .bash_rc or .zshrc or whatever config file you typically use for your shell.
PATH=$PATH:~/.cargo/bin
The backend server is a rust application that runs on top of diesel and hyper. You'll need the diesel command line interface to get started and to create the database:
$ cargo install diesel_cli
$ diesel database setup
You'll also need to clone the fbpac-api app and run its migrations, to add a few other columns. In a separate directory:
$ git clone https://github.com/propublica/fbpac-api-public.git
$ cd fbpac-api-public
$ bundle install
$ rake db:migrate
You can kick the tires by running:
$ cd backend/server
$ cargo build
$ cargo run
This will give a server running at localhost:8080. You will also need to build the backend's static resources. To do this, in another terminal tab:
$ cd backend/server/public
$ npm install
$ NODE_ENV=development npm run watch
This will build the required static assets (javascript & css) to view the admin console at localhost:8080/facebook-ads/.
If you make any changes to the database, after you run the migration, you'll want to update the schema with diesel print-schema > src/schema.rs. You'll need to do this even if you make changes via a Rails migration.
The backend has both unit and integration tests. You will need to set up a test database alongside your actual database in order to run the integration tests. To do this, you will need to the same as above, but substitute out the database test URL:
$ diesel database setup --database-url postgres://localhost/facebook_ads_test
Note that the value for --database-url came from the TEST_DATABASE_URL value set in the .env file. Make sure that the urls match before you run the tests!
Additionally, because the integration tests use the database, we want to make sure that they aren't run in parallel, so to run the tests:
$ RUST_TEST_THREADS=1 cargo test
This will run the tests in sequence, avoiding parallelism errors for tests that require the database.
We train the classifier using python and scikit learn and the source is in backend/classifier/. We're using pipenv to track dependencies.
To download pipenv:
$ brew install pipenv
To get started you can run:
$ cd backend/classifier/
$ pipenv install
$ pipenv shell
To download the seeds for the classifier, you'll need a Facebook app with the proper permissions and you'll run the seed command like this:
$ FACEBOOK_APP_ID=whatever FACEBOOK_APP_SECRET=whatever DATABASE_URL=postgres://whatever/facebook_ads ./classify seed en-US`
Alternatively, you can build the model without seeds, relying instead just on votes in the extension and suppressions in the admin. And to build the classifier you'll want to run:
$ pipenv run ./classify build
To classify the ads you've collected you can run:
$ pipenv run ./classify classify
You can download pre-trained models with pipenv run ./classify get_models.
Translations for the extension are stored in extension/_locales/${locale}/messages.json.
A locale is a ISO 639-1 language code (e.g. en, de) with an optional ISO 3166-1 Alpha-2 country suffix (e.g. de_CH).
Users can select from all known languages and countries while onboarding. The UI then uses the first available translation in following order: ${langauge}_${country}, ${langauge}, en.
In extension/src/i18n.js a list of active language and country codes can be defined. Active ones get prioritised in the UI.
You can customize, for example font sizes, with [lang] and [data-locale] CSS selectors:
[lang=de] .toggle {
font-size: 0.78rem;
}
[data-locale=de_CH] .toggle {
font-size: 0.78rem;
}- Help Us Monitor Political Ads Online
- Mehr Transparenz im Schatten-Wahlkampf
- Bringen Sie Licht in den dunklen Facebook-Wahlkampf
- Wie werben die Parteien auf Facebook?
- So werben die Parteien auf Facebook
- Warum Zuckerberg den deutschen Wahlkampf durchleuchten ließ
- Trust Issues
- Facebook Allowed Questionable Ads in German Election Despite Warnings
- Versuch der anonymen Einflussnahme auf den Bundestagswahlkampf
- Bundestagswahl: Versuch anonymer Einflussnahme
- Same-sex marriage survey: help us track targeted ads on Facebook
- Revealed: how Australians are targeted with political advertising on Facebook (Searchable Database)
- Adani posts weird video ad on Facebook to fend off Carmichael criticism
- How Malcolm Turnbull, GetUp and Adani are using Facebook ads to push their agenda
- Hjælp os med at kortlægge politiske reklamer på Facebook
- Facebook Allowed Political Ads That Were Actually Scams and Malware
- Political Ads on Facebook
- Helfen Sie uns, verdeckte Polit-Werbung zu enttarnen
In general, the project needs more tests. We've written a couple of tests for parsing the Facebook timeline in the extension directory, and a few for the tricky bits in the server, but any help here would be great!
Also, the rust backend needs a bit of love and care, and there is a bit of a mess in backend/server/src/server.rs that could use cleaning up.