NUL in category paths causes server error

[John Vandenberg]

Arguably %00 should be handled in DRF or even Django, and maybe it is and I need to enable some voodoo setting.

2020-07-31T09:56:35.137157Z [info     ] request_started                [django_structlog.middlewares.request] ip=127.0.0.1 request=<WSGIRequest: GET '/oscarapi/categories/%5D%00/'> request_id=e4339ddf-31c0-42df-91d1-ed6da6803ba4 user_agent=schemathesis/2.3.1 user_id=None
2020-07-31T09:56:35.142889Z [info     ] AnonymousUser                  [permissions] ip=127.0.0.1 request_id=e4339ddf-31c0-42df-91d1-ed6da6803ba4 user_id=None
A string literal cannot contain NUL (0x00) characters.
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/rest_framework/views.py", line 493, in dispatch
    self.initial(request, *args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/rest_framework/views.py", line 411, in initial
    self.check_permissions(request)
  File "/usr/local/lib/python3.6/site-packages/rest_framework/views.py", line 332, in check_permissions
    if not permission.has_permission(request, self):
  File "/usr/local/lib/python3.6/site-packages/drf_composable_permissions/p.py", line 32, in f
    return self._run_operator(self.operator, f1, f2)
  File "/usr/local/lib/python3.6/site-packages/drf_composable_permissions/p.py", line 18, in _run_operator
    return f1() or f2()
  File "/usr/local/lib/python3.6/site-packages/drf_composable_permissions/p.py", line 31, in f2
    return getattr(self.c2, name)(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/drf_composable_permissions/p.py", line 32, in f
    return self._run_operator(self.operator, f1, f2)
  File "/usr/local/lib/python3.6/site-packages/drf_composable_permissions/p.py", line 14, in _run_operator
    return f1()
  File "/usr/local/lib/python3.6/site-packages/drf_composable_permissions/p.py", line 28, in f1
    return getattr(self.c1, name)(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/rest_framework/permissions.py", line 212, in _queryset
    queryset = view.get_queryset()
  File "/usr/local/src/django-oscar-api/oscarapi/views/product.py", line 120, in get_queryset
    return find_from_full_slug(breadcrumb_path, separator="/").get_children()
  File "/usr/local/src/django-oscar-api/oscarapi/utils/categories.py", line 72, in find_from_full_slug
    categories = create_from_sequence(category_names, False)
  File "/usr/local/src/django-oscar-api/oscarapi/utils/categories.py", line 19, in create_from_sequence
    root = Category.objects.get(depth=1, slug=slug)
  File "/usr/local/lib/python3.6/site-packages/django/db/models/manager.py", line 82, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 402, in get
    num = len(clone)
  File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 256, in __len__
    self._fetch_all()
  File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 1242, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
  File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 55, in __iter__
    results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
  File "/usr/local/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1140, in execute_sql
    cursor.execute(sql, params)
  File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 67, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 76, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
ValueError: A string literal cannot contain NUL (0x00) characters.
2020-07-31T09:56:35.150982Z [info     ] request_finished               [django_structlog.middlewares.request] code=500 ip=127.0.0.1 request=<WSGIRequest: GET '/oscarapi/categories/%5D%00/'> request_id=e4339ddf-31c0-42df-91d1-ed6da6803ba4 user_id=None
Internal Server Error: /oscarapi/categories/]�/
"GET /oscarapi/categories/%5D%00/ HTTP/1.1" 500 169

[John Vandenberg]

fwiw, I have done full runs of schemathesis for anon and logged in user, and other than #227 which is Sqlite specific , /categories/{breadcrumbs}/ is the only endpoint where I encountered this type of problem in Oscarapi. IMO that is pretty good.

[Martijn Jacobs]

I will check this to see if oscarapi need to do some more validation in utils/categories.py. Keep you posted!

[Martijn Jacobs]

@jayvdb I'm not really able to reproduce the problem locally (or in a testcase). Curl shows me what I expect it will show:

curl localhost:8000/api/categories/%5D%00/
{"detail":"No Category found matching the query"}

Can you provide me which database backend you use and the details of the versions of:

• django
• django-rest-framework
• django-oscar
• oscarapi

And if you are able to reproduce this, could you:

• Provide me the steps how to do that, or
• Add a pdb on this line and post the exact value of breadcrumb_str as a comment in this issue?

[Martijn Jacobs]

@jayvdb Any chance you could provide me some info so I can reproduce this?

[Martijn Jacobs]

Please reopen when you found a way to reproduce this as requested above.