Giters

aio-libs / janus

Thread-safe asyncio-aware queue for Python

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Performance benefits?

jorenham opened this issue · comments

commented

So I've been testing the performance a bit, varying e.g.

  • 1:1, 16:1, 64:1 producer : consumer ratio
  • normal/lifo/priority queue etc
  • asyncio/uvloop event loop
  • async -> async, async -> sync, sync -> sync
  • (probably more)

I found that, janus queues are ~5x slower in sync->sync, ~9x slower in sync->async, and ~15x slower in async->async. This is pretty much consistent across all parameter sets.

This confirmed my suspicion that the performance gain of parallel computation is often less than the cost of using e.g. threading.Lock a lot (the GIL certainly doesn't help either).

Right now, I can imagine that many users have incorrect expectations of janus. To avoid this, you could add an example that shows how janus can outperform single-threaded asyncio, by employing multiple threads. Additionally, a caveat about janus' performance would be helpful.

commented

Interesting, can you clarify what you're comparing to in each of these cases?

A cursory glance over Janus' code which I have't really looked at before shows that there'd be some pretty easy performance gains by splitting Janus up into different queue types depending on the sender/receiver thread types. That'd be harder to both use and maintain so might be a hard sell.

commented

This library is about sync/async and async/sync communication, there is price to pay for extra synchronization. Use appropriate queue for your case.

(Also without benchmarks it is hard to discuss this issue in the first place, not clear what are you measuring.)

commented

I fully agree that it's very difficult to correctly bridge async <-> sync code, and that Janus attempts to solve this in the context of producers - consumers, with a queue as medium.

The producer/consumer pattern is often used for performance reasons, e.g. map-reduce, wsgi, websockets, etc.

Janus is presented as a generic solution for bridging async/sync code using the producer/consumer pattern, and provides no specific use-cases, or one of those "what is this project / what this project is not" sections.

This makes is easy to think it's a good idea to use Janus in your a performance-sensitive project.

But my quick-and-dirty benchmarks showed that, in a typical producer / consumer context, the Janus queue is significantly slower than conventional queues, likely overshadowing the performance that can be gained from employing the producer-consumer pattern altogether.

So, considering the amount of users, I believe it is very important that describe in the readme:

  • the precise problem that Janus aims to solve,
  • an realistic situation where using Janus is better (cleaner code, no significant performance impact) than the non-Janus alternative,
  • when it is not a good idea, (e.g. high-throughput performance bottleneck), and
  • transparant benchmarks.
commented

Library was created exactly for reason stated in read:

Mixed sync-async queue, supposed to be used for communicating between classic synchronous (threaded) code and asynchronous (in terms of asyncio) one.

I do not think we ever calmed any performance gains, or anything. I agree that docs could be better and we are happy to accept any contributions there.