django-trello-webhooks
Django application for managing Trello webhooks.
*LOOKING FOR CONTRIBUTORS*
UPDATE 01-Dec-2014
The application can now be deployed directly to Heroku using their Deploy button. In addition, the test app that is bundled into this repo has been deployed and is live.
- It is listening to a public Trello board
- Updates are posted to an open HipChat room
- You can log in to the admin site using the username public and the password trello
I am going to post further docs on the Trello board itself, so that will become the primary source for updates: https://trello.com/b/TAAnwdP9/
Status
The app is now working, and deployable to Heroku (see below). The main outstanding issue is writing some proper docs, but until / unless there is any genuine external interest shown in this project I won't be spending any formal time on it, so please don't expect much more.
Background
Trello is a very simple web app that manages lists of cards on a board - it's like a digital pinboard. You can create cards, comment on them, add attachments, move them from list to list. To make it possible to integrate with Trello, it fires off callback events whenever a user interacts with any entity within it. There is a huge list of available events. This project is designed to make it easy to capture those events, and it includes a canonical implementation of an event receiver that pipes those events to HipChat, which is a hosted IRC-like chat app - exactly like Slack.
Trello webhooks can only be managed programmatically via the API, which is clever, but also a massive PITA. In addition, whilst creating a new webhook, Trello will immediately call the registered callback URL with a HEAD request to verify that it exists. This makes setting up webhooks fiddly and complex (esp. when experimenting in a local development environment).
This project is a Django app that makes managing Trello webhook easy. Well, easier. It exposes Trello webhooks through the Django admin site as standard Django models that can be created and deleted through a web UI. Incoming webhook events are then exposed via Django signals that are raised, and can be integrated into your application.
How does it work?
Each webhook is modelled as a Django model (Webhook
), and each callback
received on the webhook is modelled as a CallbackEvent
object that
contains the JSON payload (so it will maintain a complete history of all
callbacks).
The Webhook.save()
method is used to register the webhook with the Trello
API. The corresponding delete method is used to delete the webhook from Trello.
The app contains a single view function, api_callback
, which receives the
callback from Trello, and which also supports the synchronous activation of
new webhooks by Trello. (When you create a webhook via the Trello API, they
will immediately issue a HEAD request to the callback url supplied, so you
need to be able to handle this immediately.)
The important bit is then how you use the callback in your application.
This is done via Django signals. On each webhook callback, the app sends the
callback_received
signal, passing in the data received via the callback.
Your application then connects via this signal; below is taken from the included test_app, which sends the formatted event to HipChat:
from django.conf import settings
from django.dispatch import receiver
from test_app.hipchat import send_to_hipchat
from trello_webhooks.signals import callback_received
@receiver(callback_received, dispatch_uid="callback_received")
def on_callback_received(sender, **kwargs):
if settings.HIPCHAT_ENABLED:
event = kwargs.pop('event')
send_to_hipchat(event.render())
If you wanted to filter out only certain events for sending:
def on_callback_received(sender, **kwargs):
event = kwargs.pop('event')
if event.event_type == 'commentCard':
send_to_hipchat(event.render())
There is a Django management command which can be used to synchronise any
existing webhooks (in both directions), called sync_webhooks
. Run on
its own, without any arguments, this will look up all the webhooks in
the local Django database, and sync then with Trello (creating them if
they don't already exist). It will also check Trello for any webhooks
that it has registered that do not exist locally, and create them.
Rendering the payload
Once you've received a callback, along with its JSON payload, the next
question is how to use it effectively. It is assumed (by me) that the
core use case for this project is to pipe the events elsewhere - in
our case it's to HipChat, but other messaging services are available -
you've could even go old-school and email people stuff. Whatever you
decide to do, you will probably want to convert the JSON into some
form of readable text output. In order to facilitate this each event
type (createCard
, commentCard
etc.) has an associated Django
template. The CallbackEvent
model has a render
method that
passes the entire JSON payload into the relevant template as the
template context, so that you can extract the data.
Below is an example of the default commendCard.html
template.
<b>{{action.memberCreator.fullName}}</b> commented
on the card "<b>{{action.data.card.name}}</b>"
on the board "<b>{{action.data.board.name}}</b>":
<blockquote>{{action.data.text}}</blockquote>
The default templates are designed to show what is possible - and it's
recommended that you override these in your application. You can do
this using Django template overriding - add your template to your
application in the same locaion (/templates/trello_webhooks/<event_type>.html
)
and declare your app above the trello_webhooks
app in the
INSTALLED_APPS
setting, and your template will be used instead
of the default.
The combination of overrideable templates and the callback_received
signal
mean that you should be able to integrate Trello fully into your app.
NB One word of caution
I have made no attempt to ensure that all events are covered - that's not
really the point. This app will store and forward any event that it
receives. In order to make it a little easier to manage unexpected events
there is a property of the CallbackEvent
that is displayed in the
admin site list view - Has Template. If this is True, then this is
an event for which we have a default template. If it's False, then
this is a new one on us - and you are encouraged to play around with
adding a new template. Do please feed all new default templates back
to the project.
Configuration
There are three mandatory environment settings (following the 12-factor app principle):
- TRELLO_API_KEY
- TRELLO_API_SECRET
- CALLBACK_DOMAIN
The first two are the core Trello developer API keys - available from here: https://trello.com/1/appKey/generate
The CALLBACK_DOMAIN is included as you need to give a fully-qualified domain to the Trello API, and it's not always possible to infer what that might be - for instance when developing locally, you will need a tunnel from your machine out onto the web using something like ngrok.
When managing hooks via the Trello API a third key is required, and this is
user specific - the admin site has a link next to the auth_token field on
the form for creating a new Webhook. This uses the Trello API client.js to
perform the Oauth dance - and supplies the user token. All webhooks are
registered against a user token. That's how it works. (NB you can pass any
user tokens you have lying around to the sync_webhooks
command and it
will check Trello for any existing webhooks registered with those tokens.)
Tests
You can run the tests yourself in the normal manner:
$ python manage.py test
However, if you have tox
installed (and I'd really recommend you do),
then you can simply run $ tox
, and this will also include coverage.
Coverage isn't 100% (when is it), but if you do contribute please do include tests for any changes that you make.
The tests themselves use mock objects to replicate the two Trello API calls
(list_hooks
and create_hook
), so no internet access is required. (The
project relies on py-trello, and
that has coverage for the API calls.)
Setup
The app is available on PyPI as django-trello-webhooks
, so install with pip
:
$ pip install django-trello-webhooks
Further Developments
- Write some tests
- Better integration with the Trello API
- Handle user auth token expiry properly
- Integration with Heroku's "Deploy to Heroku" button
Contributing
Usual rules apply - fork, send pull request. Please try and adhere to the existing coding style - it may not be your style, but it's the project's style, so PRs will be rejected if they 'smell bad'. Specifically, given that this is an app that is pushing data over the wire, and therefore hard to debug - lots of logging, and lots of comments. Seriously. Lots.
Licence
MIT (see LICENCE file)
Dependencies
The core Trello API integration is done using py-trello from Richard Kolkovich (@sarumont), so thanks to him for that. He naturally relies on requests from Kenneth Reitz, as well as request-oauthlib, so thanks to anyone involved with either of those.
Addenda
The webhook API works on the concept of a Trello model id. This refers to the object being watched - and could be a Board, a List, a Card etc. Getting these ids is a bit of a pain, to put it mildly, so I would strongly recommend using the excellent Trello Explorer app from Harald Wartig (@hwartig).
I would also recommend the use of ngrok to expose your local Django dev server during development.
As for development itself - use virtualenv, install dependencies from requirements.txt and set up environment variables. If that doesn't mean anything to you - I'm afraid you have a lot to learn.
Deploying to Heroku
TODO: write proper docs
This repo contains a test app can be deployed directly to Heroku using their Deploy button. This app will pipe Trello updates directly to a Hipchat room. You will need the following information in order to set up and configure the app:
TRELLO_API_KEY
, TRELLO_API_SECRET
, which you can get from here - https://trello.com/1/appKey/generate
HIPCHAT_API_TOKEN
, HIPCHAT_ROOM_ID
, which you can get from hipchat.com
In addition, you will need to set the CALLBACK_DOMAIN
environment setting once the app
has been deployed. This should be set to the <app_name>.herokuapp.com domain, that is
available once Heroku has deployed it.
The recommended hacking method (IMO) is to set up the Heroku app, and use that as your main git remote - pull it down locally, change the relevant templates, push back to Heroku. If you're actually adding functionality, then please follow the contributing instructions above.