Dynamic order by in paginated response decorator
tharrington opened this issue · comments
How would I add order by the the decorator?
For example:
@posts.route('/posts', methods=['GET'])
@authenticate(token_auth)
@paginated_response(posts_schema, order_by=Post.timestamp,
order_direction='desc',
pagination_schema=DateTimePaginationSchema)
def all():
"""Retrieve all posts"""
return Post.select()
However, I want the client to pass in a field name for the order by. If this happens, the client is really only passing in a string so the decorator errors because there is no desc attribute on a string:
order_by = pagination.get('order_by')
order_direction = pagination.get('order_direction')
if order_by is not None:
o = order_by.desc() if order_direction == 'desc' else order_by
I am certainly new to flask and python generally, so I apologize if this solution is obvious. Ultimately I want an elegant solution that will allow this paginated response to be flexible across many different tables.
Thanks again!
If you have a field name in a string, you can get the actual attribute of that name as follows:
attribute = getattr(Model, field)
But note that this is considered a bad practice unless you first validate the field to make sure it is one of a list of accepted values. You can't let the client send any field name that it likes.
Thanks for your response. The client is a react app with a datatable of users. I would assume I could just user the getattr (or perhaps some other method if one exists to check for column names which would allow the api to throw an error if a bad value was passed) method you mentioned and pass the model name from the api endpoint that utilizes the decorator. Then wrap the order by part of the decorator in a try/catch... for example:
@paginated_response(posts_schema, model='User',
order_direction='desc',
pagination_schema=DateTimePaginationSchema)
decorator:
if order_by is not None:
o = getattr(model, order_by)
o = o.desc() if order_direction == 'desc' else order_by
If this is a bad approach, would you just recommend doing this in the api file outside the decorator? This is how I implemented it for search on the user api:
@users.route('/users', methods=['GET'])
@authenticate(token_auth)
@paginated_response(users_schema)
def all():
"""Retrieve all users"""
args = request.args
if args is not None and 'query' in args:
search = "%{}%".format(args.get('query'))
return User.select().where(User.searchable.like(search))
return User.select()
You can't let the client send any field name that it likes.
I take this to mean that a client can pass in whatever they like so long as errors are handled accordingly.
Neither approach is good, unfortunately.
The problem with the first approach is that you are letting the client pick any field that it wants. What happens if the client sets order_by='password', for example? This isn't going to leak passwords, but I hope you agree this is still a bit worrying. At the very least the client will know that you have an attribute called password if the call succeeds. They can start sending random words and use the success vs error response to determine what attributes you have, even those that aren't intended to be public. For this approach to work you have to create a list of allowed orderings, and validate that the requested order is one from the list. And this should be in the schema, so that your documentation also shows the valid sorting options.
The second approach is bad because you are bypassing validation/documentation and handling the query arguments directly, without going through schemas.
They can start sending random words and use the success vs error response to determine what attributes you have, even those that aren't intended to be public. For this approach to work you have to create a list of allowed orderings, and validate that the requested order is one from the list. And this should be in the schema, so that your documentation also shows the valid sorting options.
Please correct me if I'm wrong, but this would mean I would ultimately need to get rid of this schema:
class StringPaginationSchema(ma.Schema):
class Meta:
ordered = True
limit = ma.Integer()
offset = ma.Integer()
after = ma.String(load_only=True)
count = ma.Integer(dump_only=True)
total = ma.Integer(dump_only=True)
@validates_schema
def validate_schema(self, data, **kwargs):
if data.get('offset') is not None and data.get('after') is not None:
raise ValidationError('Cannot specify both offset and after')
In favor of multiple pagination schemas in order to provide for the sorting of tables by their unique column names:
class UserPaginationSchema(ma.Schema):
class Meta:
ordered = True
limit = ma.Integer()
offset = ma.Integer()
after = ma.String(load_only=True)
order_by = ma.String(load_only=True)
order_direction = ma.String(load_only=True)
search = ma.String(load_only=True)
count = ma.Integer(dump_only=True)
total = ma.Integer(dump_only=True)
@validates('order_by')
def validate_order_by(self, value):
if value != 'first_name':
raise ValidationError('You can only sort by the first name!')
@validates_schema
def validate_schema(self, data, **kwargs):
if data.get('offset') is not None and data.get('after') is not None:
raise ValidationError('Cannot specify both offset and after')
You can separate the pagination from the sorting and use two decorators and two schemas. You can also pass the list of allowed orderings as an argument into the decorator, which in turn passes it as an argument into the schema.
I've gone with this approach:
if order_by is not None:
order_by_dict = dict(order_by=order_by)
try:
order_schema.load(order_by_dict)
order_by_attr = getattr(model, order_by)
o = order_by_attr.desc() if order_direction == 'desc' else order_by_attr
select_query = select_query.order_by(o)
except ValidationError as err:
return {'errors': err.messages}, 400
class UserOrderBySchema(ma.Schema):
order_by = fields.Str(validate=validate.OneOf(["first_name", "last_name"]))
This doesn't quite get me there in terms of documentation, but it is functional and secure
Looks much much better. :)
If you want to keep hacking on it, I think what's missing here in terms of docs is to include the sorting schema in an @arguments decorator, so that the documentation system can pick it up. This is easier said than done, but you can see how the pagination decorator in this repo calls @arguments internally to register its own query arguments with the docs.
Once you do this, you will not need to manually load and validate your schema, this is all going to be done for you.
@miguelgrinberg Could you explain in steps if you be so kind, on how to implement this? (sorting the posts from passing url parameters), I am farely new to flask api development, cant fully grasp how to implement what's discussed above
@anonhacker47 I recommend that you study my paginated response decorator. The solution for sorting uses the same techniques. This is fairly advanced stuff, though. You may want to try to implement this without additional decorators first, just by adding your sorting in an @arguments decorator and then handling the sort at the start of the endpoint code.