willkg / jansky

a standalone rewrite of the socorro processor. intended to work with antenna. pre-alpha.

Processor: A Breakpad crash report processor

Built to accept crashes collected by Antenna, applies JSON/dump pairs to the stackwalk_server application, parses the output, and records the results. The processor, coupled with stackwalk_server, is computationally intensive. Multiple instances of the processor can be run simultaneously.

Free software:	Mozilla Public License version 2.0
Documentation:	https://processor.readthedocs.io/
Status:	Pre-alpha

Quickstart

This is a quickstart that uses Docker so you can see how the pieces work. Docker is also used for local development of Processor.

For more comprehensive documentation or instructions on how to set this up in production, see docs.

1. Clone the repository:

   $ git clone https://github.com/mozilla/processor

2. Install docker 1.10.0+ and install docker-compose 1.6.0+ on your machine

3. Download and build Processor docker containers:

   $ make build

   Anytime you want to update the containers, you can run make build.

4. Run with a prod-like fully-functional configuration.

   1. Running:

      $ make run

   2. Verify things are running:

      In another terminal, you can verify the proper containers are running with docker ps.

   3. Look at runtime metrics with Grafana:

      The statsd container has Grafana. You can view the statsd data via Grafana in your web browser http://localhost:9000.

      To log into Grafana, use username admin and password admin.

      You'll need to set up a Graphite datasource pointed to http://localhost:8000.

   4. Shutting down the processor:

      When you're done with the Processor process, hit CTRL-C to gracefully kill the docker container.

   If you want to run with a different Processor configuration, put the configuration in an env file and then set PROCESSOR_ENV. For example:

   $ PROCESSOR_ENV=my.env make run

   See prod.env and the docs for configuration options.

5. Run tests:

   $ make test

   If you need to run specific tests or pass in different arguments, you can run bash in the base container and then run py.test with whatever args you want. For example:

   $ make shell
   app@...$ py.test
   
   <pytest output>
   
   app@...$ py.test tests/unittest/test_crashstorage.py

   We're using py.test for a test harness and test discovery.

For more details on running Processor or hacking on Processor, see the docs.