snok / django-guid

Inject an ID into every log message from a Django request. ASGI compatible, integrates with Sentry, and works with Celery

Home Page:https://django-guid.readthedocs.io

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

ASGI support for later Django versions (works with Django 3.0.x)

JonasKs opened this issue · comments

With ASGI being stable and more async support to Django is coming up, this package eventually have to change it's design to be ASGI compliant.

  • ASGI by protocol can handle multiple requests on one thread. Every time await is used, the control of the loop will be handed back to the event loop, queuing the resumption of the function. The next function will run until it calls on an await, and then the entire thing repeats. This means that a thread will by design be able to handle multiple requests.

  • Most of Django do not support awaitat the moment, so django-guid will work on ASGI today. The way it works right now is that when it gets an ASGI request on the event loop, it dispatches it to a thread pool where each thread again handles one request at the time.

Essentially this means that in the future, in order to keep this package living we have to find another ID to use. Luckily the Django people has been kind to us and implemented a replacement for us.

I will look into implementing this ASAP.

Thanks to knbk on #freenode.

I did create a asgi branch, wrote a simple asgi application and started testing out the asgiref api. How ever, it didn't behave as I expected and while there is a readme, the API isn't really documented or got any examples. That being said, I'll continue looking into this soon as Django 3.1 is getting closer.

I've found the solution for this. Will be implementing this the following week.

Sorry for few updates on this one, but I can assure that it's being worked on. There's been a few obstacles, but design, pattern and implementation is about to be finished.

I've also been in touch with Andrew Godwin regarding a potential memory leak.

I know I haven't commited any code for anyone to look at, but I want it to be a bit more finished and tested before I let people see. Right now it's not organized, not documented and not pretty.

I've tested the memory consumption of uvicorn with and without the async Django-GUID middleware(as the only middleware). It does not matter if I send 1000 requests or 100000, the memory consumption difference is always about 0.13MB. There is no memory leak, only a bit more memory consumption.

# 100k requests with Django-GUID async supported middleware
# start memory
353408K
# 100k requests later
670792K

# 100k requests _without_ Django-GUID async supported middleware
# start memory
353400K
# 100k requests later
670924K

EDIT: Forgot to write that a GUID is 16 byte, so 100k requests would be 1.6 MB - which is 1.3MB more than the actual memory consumption.

Busy weeks. Getting there though.
Sync middleware works as expected, all tests pass. Looking into how to properly write tests for the async middleware now. Release should not be too far away. :)

Coverage is at 100%, docs are almost done.