linuxlewis / django-diffs

A django app that allows users to register models to cache their changes for a fixed time.

Django Diffs

Django diffs allows models to be registered to cache it's changes (or diffs) over a fixed time period.

The diffs are stored in redis using a SortedSet and accessed via a manager-like object on the registered django model class.

It's compatible with Python 2/3 and Django 1.8 and above. It requires an available redis server.

Table of Contents

• Getting Started
• Configuration
• Pruning Diffs
• Custom Serialization
• Related models

How does it Work?

Models are registered with the @diffs.register decorator and their changes are serialized and saved to redis on signals. The decorator installs django-dirtyfields to the model on registration to get the changed fields of the model instance.

Changes can be accessed via the diffs manager on the registered model. The diffs manager returns a list of Diff objects that have properties of data, created, and timestamp.

The manager can be accessed via the class like Question.diffs or like a related manager on the instance instance.diffs.

Here's a quick example.

# models.py

import diffs

@diffs.register
class Question(models.Model):
    question_text = models.CharField(max_length=200)
    pub_date = models.DateTimeField('date published')

question = Question.objects.create(question_text='What is life?')

question.question_text = 'What is python?'
question.save()

for diff in question.diffs:
    print(diff.timestamp)
    print(diff.data)
    print(diff.created)

diffs = Question.diffs.get_by_object_id(question.id)

Why?

You need to cache the changes to a single django model or collection of models for a fixed time period.

Tracking the changes prevents clients from having to re-request all of the model data which is assumed to be costly.

Getting Started

• Add django-diffs to requirements.txt

pip install django-diffs

• Add diffs to INSTALLED_APPS

INSTALLED_APPS = (
    'diffs',
)

• Register a Model

# models.py

import diffs

@diffs.register
class Question(models.Model):
    question_text = models.CharField(max_length=200)
    pub_date = models.DateTimeField('date published')

That's it! Changes will now be tracked automatically for this model.

Configuration

Django-diffs can be configured via django.conf.settings. Below is the default configuration

# settings.py

DIFFS_SETTINGS = {
    'redis': {
        'host': 'localhost',
        'port': 6379,
        'db': 0,
    },
    'max_element_age': 60*60,
    'use_transactions': True,
    'test_mode': False
}

The following keys are supported for DIFFS_SETTINGS

redis -- A dictionary with the keys host, port and db for details of the redis server.

max_element_age -- Defines the number of seconds a single diff should be allowed to live. This is used in the pruning script to remove old elements from the set.

use_transactions -- Boolean to configure django-diffs using Django's connection.on_commit callback registry. When enabled django-diffs will defer persistence to on_commit.

test_mode -- Boolean to configure using test mode. Test mode uses fake_redis instead of real redis so a server isn't required. Use this mode when running your unittests.

Pruning Diffs

By default redis only allows you to set an expire on an entire key. You cannot set an expiry per element in a set or sorted set.

To work around this django-diffs sets the current unix timestamp as the SortedSet element score. Items can then be easily removed using the redis command ZREMRANGEBYSCORE.

All of this has been handled for you in the custom management command prune_diffs. Run this on a cron schedule to keep your cache up to date.

python manage.py prune_diffs

Custom Serialization

By default django-diffs uses django.core.serializers module to serialize the diff to json.

To use your own custom serialization format just implement the serialize_diff method on your model. It will be passed the list of dirty_fields and the created kwarg.

# models.py

import diffs

@diffs.register
class Question(models.Model):
    question_text = models.CharField(max_length=200)
    pub_date = models.DateTimeField('date published')

    def serialize_diff(self, dirty_fields, created=False):
        return {'fields': dirty_fields}

question = Question.objects.create(question_text='What will happen?')

Question.diffs.get_by_object_id(question.id)[-1].data
# {'fields': ['question_name']}

Related models

Sometimes you want to track changes on a collection of related models. These could be individual items part of a larger Report object.

Django-diffs allows you to set a parent objects by implementing get_diff_parent on the child model. It must return a model instance with an id defined.

# models.py

import diffs

@diffs.register
class Question(models.Model):
    question_text = models.CharField(max_length=200)
    pub_date = models.DateTimeField('date published')


@diffs.register
class Choice(models.Model):
    question = models.ForeignKey(Question, on_delete=models.CASCADE)
    choice_text = models.CharField(max_length=200)
    votes = models.IntegerField(default=0)

    def get_diff_parent(self):
        # save the db lookup
        return Question(id=self.question_id)


question = Question.objects.create(question_text='What will happen?')
choice = Choice.objects.create(choice_text='Nothing', question=question)

choice.choice_text = 'Something'
choice.save()

# returns diffs for question and it's choices
len(question.diffs) # 3