Description

ProjectMonitor is a CI display aggregator. It displays the status of multiple Continuous Integration builds on a single web page. The intent is that you display the page on a big screen monitor or TV so that the status of all your projects' builds are highly visible/glanceable (a "Big Visible Chart"). ProjectMonitor currently supports:

This project is forked from pivotal/projectmonitor. CfA uses it to show the status of fellowship projects and internal tools.

Display

Just open a browser on /. The page refreshes every 5 minutes with the latest status fetched by the cron job or received via Webhook. That is, refreshing the page doesn't cause the individual builds to be re-polled.

Layout

The layout consists of a grid of tiles representing the projects. The number of projects that need to be displayed is determined automatically.

Tile colors

Tiles are green for green projects, red for red projects. If the build server is online but no builds have been run then the tile will appear in yellow.

Build Statuses

Each project lists the amount of time since the last build in the bottom left corner. Underneath that is the build status history. The last 5 builds are displayed from left to right, in reverse chronological order -- the most recent build will be on the left and the least recent on the right. Successful builds are marked with a check mark, and unsuccessful builds are marked with an x.

Readme Statuses

For a project's readme to be valid, it must have an installation section or has a section indicating the project has been moved. A valid installation section includes a head with the root word of "Install", "Setup", "Build" or "Deploy". A section indicating the project has moved must include the phrase "Repository has moved". A project can have no readme and still be considered valid if the project is less than a week old.

Admin Interface

Click 'manage projects' at the lower left to log in then add or edit projects.

Add projects

After logging in, click the "New Project" button and enter the details for a build you want to display on ProjectMonitor. The "Name" and "Project Type" are required. You will need to either connect your service via Webhooks or polling.

To connect via Webhooks, the project settings page will display the Webhook URL you'll need to enter in your CI instance's settings. The Webhook URL isn't generated until after a project is created, so you'll need to select the 'Webhooks' radio button, click 'Create', and then click the 'Edit' button for the newly-created project to get the Webhook URL.

In order for Webhooks to work, you'll also need to make sure that the machine hosting ProjectMonitor is accessible by the machine hosting your CI instance.

If you want to set up a project to connect via polling instead, you'll typically need to enter the base URL, build name or ID, and your login credentials with the CI service.

TeamCity

For TeamCity projects, find the buildTypeId (usually something like 'bt2') from the URL, which should look like one of the following:

http://teamcity:8111/app/rest/builds?locator=running:all,buildType:
http://teamcity:8111/viewType.html?buildTypeId=
http://teamcity:8111/viewLog.html?buildId=1&tab=buildResultsDiv&buildTypeId=

You will also need a valid user account and password.

If you want TeamCity to connect via Webhooks, you'll need to install the TcWebHooks plugin on your TeamCity instance. When setting up the webhook in TeamCity, make sure the payload format is set to "JSON" (it might show up as "JSON (beta)").

If you want to connect to TeamCity via polling, you will need to ensure that your TeamCity instance is accessible by the machine running ProjectMonitor.

Semaphore

When configuring Semaphore, you should use the Branch History URL from the API section of your Project Settings page.

This ensures that no build statuses will be missed.

If you notice that there are build statuses missing in project monitor, ensure that you are NOT using the Branches URL from the API section (vs. the recommended Branch History URL). The Branches URL from the API section returns only the latest build status, instead of the history, so if builds occurred between status fetches, they would be missed and not be reflected in project monitor.

Jenkins

If you want Jenkins to connect via Webhooks, you will need the Jenkins notification plugin.

If you want to connect to Jenkins via polling, you'll need to ensure that your Jenkins instance is accessible by the machine running ProjectMonitor.

Travis

If you want Travis to connect via Webhooks, you will still need to enter the GitHub account, repository, and optionally branch name for the codebase being built in Travis.

TDDium

TDDium only supports connecting via polling, not Webhooks.

In order to get polling configured, you will need to log in to your TDDium dashboard, go to Organizations using the drop down in the top right corner. Then click on organization settings for the appropriate organization. Then click on "Chat Notifications"; CCmenu is at the bottom of the page. That should take you to a URL that looks like:

https://api.tddium.com/cc/SOME-TOKEN-HERE/cctray.xml

The value for "SOME-TOKEN-HERE" is the TDDium authentication token you'll need to paste into the ProjectMonitor settings. The XML returned by that link will look something like:

<Projects>
   <Project name="foobar (master)" webUrl="https://api.tddium.com/1/reports/151751" lastBuildLabel="151751" 
   lastBuildTime="2013-01-08 18:20:05" lastBuildStatus="Failure" activity="Building"/>
</Projects>

The "TDDium Project Name" field in the ProjectManager settings will need to be set to the full value of the Project name attribute, complete with the branch name in parentheses (in this case, "foobar (master)").

Installation

Get the code

ProjectMonitor is a Rails application. To get the code, execute the following:

git clone git://github.com/pivotal/projectmonitor.git
cd projectmonitor
bundle install

Initial Setup

We have provided an example file for database.yml. Run the following to automatically generate these files for you:

rake setup

You likely need to edit the generated files. See below.

Set up the database

You'll need a database. Create it with whatever name you want. If you have not run rake setup, copy database.yml.example to database.yml. Edit the production environment configuration so it's right for your database:

cp config/database.yml.example config/database.yml
<edit database.yml>
RAILS_ENV=production rake db:create
RAILS_ENV=production rake db:migrate

Set up memcached for performant/distributed caching

Statuses are cached via an in-memory store by default. If you want to use memcache, you'll need to ensure it's installed. It should be very easy to install if you have homebrew or another package manager:

brew install memcached

After you have successfully installed memcached follow the instructions to run it.

Authentication support

Password authentication

Project monitor uses Devise to provide both database backed authentication and Google OAuth2 logins.

Regular password authentication for managing project settings is enabled by default and can be switched off by setting the password_auth_enabled setting to false. To ensure strong password encryption you should adjust the value for password_auth_pepper and password_auth_stretches appropriately.

Google OAuth2 setup

To use Google OAuth2 authentication you need Google apps set up for your domain and the following configuration options specified:

oauth2_enabled: true
oauth2_apphost: 'MY_APP_ID'
oauth2_secret: 'MY_SECRET'

Setup Cron with Whenever

We have included a sample whenever gem config in config/schedule.rb. Refer to the whenever documentation for instructions on how to integrate it with your deployment. Refer to Heroku scheduler documentation for instructions on how to integrate the rake task with your Heroku deployment.

The default schedule clears log entries and fetches project statuses every 3 minutes.

The fetch project task is what goes out and hits the individual builds. We find that if you do this too frequently it can swamp the builds. On the other hand, you don't want ProjectMonitor displaying stale information. At Pivotal we set it up to run every 3 minutes.

Start workers

The cron job above will add jobs to the queue, which workers will execute. To start running the workers, use the following command:

rake start_workers

The default number of workers is 2, but if you wanted 3 you would call it like this:

rake start_workers[3]

These workers need only be started once per system reboot, and must be running for your project statuses to update. To stop the workers, run this command:

rake stop_workers

The workers are implemented using the delayed_job gem. The workers are configured to have a maximum timeout of 1 minute when polling project status. If you want to change this setting, you can edit config/initializers/delayed_job_config.rb

Start the application

Execute:

nohup rails server -e production &> projectmonitor.log

Configuration

Each build that you want ProjectMonitor to display is called a "project" in ProjectMonitor. You can log in to set up projects by clicking the "Manage Projects" link in the bottom-right corner of the main ProjectMonitor screen. You can either create a user using the console as follows:

rails c production
User.create!(login: 'john', name: 'John Doe', email: 'jdoe@example.com', password: 'password', password_confirmation: 'password')

Or, if you have set up Google OAuth2 as per above, you can simply log in with Google to create a new user account.

Importing and Exporting Configurations

You can export your configuration for posterity or to be transferred to another host:

rake projectmonitor:export > ${your_configuration.yml}

Or using heroku:

heroku run rake projectmonitor:export --app projectmonitor-staging > ${your_configuration.yml}

Or you can download it using the configuration endpoint, using curl (or your web browser):

curl --user ${username}:${password} ${your_project_monitor_host}/configuration > ${your_configuration.yml}

NOTE: That heroku doesn't treat STDERR and STDOUT differently so you may get some warnings at the beginning of the generated file that you'll have to remove manually.

It can be imported in a similar way:

rake projectmonitor:import < ${your_configuration.yml}

On heroku or another host which doesn't allow you to directly load files or read from stdin, you'll need to post the file to the configuration endpoint like so:

curl --user ${username}:${password} -F "content=@-" ${your_project_monitor_host}/configuration < ${your_configuration.yml}

Deployment

Heroku

To get running on Heroku, after you have cloned and bundled, run the following commands:

NB: These instructions are for the basic authentication strategy.

heroku create
git push heroku master
heroku run rake db:migrate
heroku config:add REST_AUTH_SITE_KEY=<unique, private and long alphanumeric key, e.g. abcd1234edfg78910>
heroku config:add REST_AUTH_DIGEST_STRETCHES<count of number of times to apply the digest, 10 recommended>
heroku run console

When inside the console, run the creating a new user step above. You should then be able to access your server and start using it.

Running tests

To run tests, run:

rake setup
rake spec

Javascript tests

To run Jasmine tests, run this once when setting up the project or if you haven't consistently been running guard:

rake jasmine:compile_coffeescript

To keep your jasmine tests current, run:

guard

Install phantomjs if needed:

brew install phantomjs

Then, to run Jasmine tests from the command line, run:

RAILS_ENV=test bundle exec rake spec:javascript

To run Jasmine tests from the browser, with a rails server running, visit http://localhost:3000/specs

See the jasmine-rails documentation for more details.

Development server

To run a local development server and worker, run:

foreman start

phae / projectmonitor