Support permission-based queryset filters

[Jeremy Bowman]

I really like how rules works for individual object permissions, but it doesn't really cover permission-based queryset filtering (which comes naturally for database-centric permission systems like django-guardian). While thinking about how to compensate for that, I thought of an extension to the API that could help fill that gap. Rather than rushing off to write a pull request, I figured I'd outline it here for feedback first. I'm envisioning something like this:

from django.db.models import Q

@rules.filter
def is_book_author(user):
    return Q(author=user)

is_book_author_or_superuser = is_book_author | rules.predicates.is_superuser

rules.add_filter('books.view_book', is_book_author_or_superuser)

Book.objects.filter(rules.q(user, 'books.view_book')

Filters would have to be defined separately from the object permission predicates, but would work very similarly; Q objects can be combined in ways that are pretty compatible with the predicate combinations already supported in rules. Existing predicates which only depend on properties of the user could be combined with Q-based filters, with predicate outcomes being represented as always-True (like Q(pk__isnull=False)) or always-False (like Q(pk__isnull=True)) Q objects.

This would also make it pretty straightforward to create a Django REST Framework filter that would use the filter associated with the correct permission:

from rest_framework.compat import get_model_name

class DjangoPermissionRulesFilter(BaseFilterBackend):

    perm_format = '%(app_label)s.view_%(model_name)s'

    def filter_queryset(self, request, queryset, view):
        user = request.user
        model_cls = queryset.model
        kwargs = {
            'app_label': model_cls._meta.app_label,
            'model_name': get_model_name(model_cls)
        }
        permission = self.perm_format % kwargs
        return queryset.filter(rules.q(user, permission))

Some of the things I like about this design:

• Keeps implementation of the permissions out of the models and model managers, so they can be grouped together with the predicate definitions
• Allows reuse of some basic filtering operations (at least within permissions on the same model or other ones with the same lookup path for the fields to compare)
• Consistency with implementing predicates for the object-based permissions
• Ability to reuse predicates that don't depend on the object in filters (this part just occurred to me and hasn't been as carefully thought through as the rest, but it seems like it should work)
• Very simple to support in Django REST Framework with only one custom filter class that can be reused for many views

Some downsides that I don't see good ways to work around yet:

• One permission can have 2 different implementations: a predicate function for a single object, or a Q object for a queryset. I really don't see any way around this without really limiting and complicating the case where you don't even need a filter for the permission (which is pretty common).
• Models with different lookup paths to the user (or related models) generally can't share filter functions; one may need Q(author=user) while another has Q(owner=user), Q(status__user=user), or even Q(creator__organization=user.organization).
• I'd kind of prefer a query filtering syntax like Book.objects.has_perm(user, 'books.view_book'), but it doesn't seem worth the effort to create a model manager mixin for it that would need to be explicitly included in all relevant models.

Thoughts? Do you think something like this would fit in rules or should go into a separate app which depends on it? And can you think of any good improvements on the API?

[dfunckt]

Yeah, I think this should go in a separate app and it actually wouldn't even need to depend on rules as far as I can see. If such a project did exist however, I'd be happy to collaborate and/or add any necessary hooks in rules for it to work.

On the idea itself, I don't really have anything of value to say, but I do like the idea of mapping dynamic Qs to simple strings (such as permissions) and don't consider the downsides you mention as blockers. I also think the filters needn't necessarily require a user, I believe such an API could work with any object being passed to a filter -- the only requirement would be for it to return a Q appropriate for the queryset at hand. You might also want to think about passing the queryset itself as a first argument too, as this could enable the filter to do more advanced stuff (like selecting only appropriate fields, doing subselects, etc).

[Jeremy Bowman]

Ok, sounds good. I'm fleshing out a proposal for best practices in handling authorization in the assorted Open edX packages and services, and this came up as a gap in the existing packages. I'll drop the proposed API in that doc and see what we can get hashed out and implemented.

I guess it would be nice to have rules as an optional dependency. I like the idea of reusing existing predicates which only depend on the user object, but you could certainly build a permission to Q object mapping without using them. May not need any specific changes in the rules codebase to accommodate that, but I'll have to see how the implementation pans out.

[Joe Davy]

Has anyone found or developed a solution to securing / filtering querysets with rules? Or if not, can anyone suggest how to best accomplish this?

[Jeremy Bowman]

Here's the proposal for a rules-based QuerySet filtering package that I mentioned above. I haven't actually needed to implement it yet, though.

[Daisy Leigh Brenecki]

I've started work on Bridgekeeper, which is a library that takes a lot of inspiration from django-rules but operates on querysets. It's currently very early days; I literally started working on this in an internal company project a few weeks ago, and extracted it into an OSS project last week, and I'm still fleshing out missing bits of documentation, as well as changing the docs around a lot to try to figure out the best way to communicate stuff. (I'm also changing the names of concepts to hopefully make it easier to understand, so don't expect the API to be stable until at least sometime after Christmas.)

To cover off a few things discussed in this thread:

• Rules (the Bridgekeeper equivalent of django-rules' predicates) all provide a filter() method (which internally uses Q objects), as well as a check() method which works a bit more like django-rules, returning a bool.
• Book.objects.visible_to(user, 'books.view_book') is something you can do in Bridgekeeper, by attaching a Bridgekeeper-supplied manager class to your models.
• You can't define arbitrary functions that depend on both the user and the object, like you can in django-rules (because of the two-different-methods restriction), but I've tried to provide enough rule classes to do most things (e.g. the is_book_author example would be Attribute('author', lambda user: user). The methods you need to override to write custom Rule subclasses, if you really need to, are also part of the documented public API.
• If you have rules that only depend on the user object, you can write them as a simple function and use the @ambient decorator to turn them into a rule object that satisfies the filter()/check() API properly.
• There's no attempt at compatibility with django-rules itself right now. However it'd be fairly simple to convert django-rules predicates which depend only on the user object into Bridgekeeper ambient rules, and I'd certainly consider adding django-rules compatibility down the track once Bridgekeeper itself has taken shape a bit more. (I should note here that I haven't actually used django-rules ever; I came across it recently, read the README, fell in love with the API, but couldn't use it because I needed QuerySet support right off the bat.)
• There's also no attempt at providing any convenience methods for Django REST Framework (although Bridgekeeper does provide a very similar QuerySetPermissionMixin for regular Django CBVs that call a .get_queryset() method)

All in all, I think what I've built is sort of similar to what @jmbowman is suggesting in this thread, although not exactly the same. I'd be pretty keen for feedback from anyone interested in this use case (but probably at https://github.com/adambrenecki/bridgekeeper/issues or [e-mail], so as not to derail this issue too much).

(PS: I hope I'm not too out of line here! I don't want to sound like I'm coming in to the issue tracker of a project that a lot of people have spent a lot of time and effort on and going "here use mine, it's better"; I'm only posting this because of the discussion in this ticket saying that this functionality belongs in a separate external library 🙂)

[James Batt]

I too was inspired by Django Rules and felt the need for filters. Before I saw this post I implemented "a django rules for filters" and it looks very similar to what @jmbowman posted above. I'm happy to report it works very nicely.

Internally at my company we've created a lib that bundles Rules and the aforementioned filters package into 1 library, along with helpers for django rest framework integration and it's been very useful.

I'd love to contribute to this project if that's what Django Rules would like to do. We really didn't want to create a packages as it'd require us to install 3 deps on every new project (1 for permissions, filters and a restframework bindings/companion). Having it all under 1 package has been very useful for allowing other devs to get started fast and read how it all works in 1 place.

[Madhav Bhagat]

@Place1 can you share your code we are working on the exact same thing... Maybe as a gist for now so we can try it out internally...

[James Batt]

@codebreach I’d be very happy too. I’ll see what I can do.

[Timo Grevers]

@Place1 any updates on your progress? I would be interested to see how it works :)

[pkariz]

@codebreach @Place1 i'm thinking of doing that, any gist just to get a feel of how complicated it is?