sheharyarn / que

Simple Job Processing in Elixir with Mnesia :zap:

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Inserting into queue is slow when queue is long

mbaeuerle opened this issue · comments

Inserting into the queue is slow when the queue is long. This is already recognizable when the queue has about 10,000 items.

This has mainly to do with the implementation of the queue. Adding items at the end of a list takes siginficant time (I think O(n^2)), because it has to reverse the list twice to update it.
By replacing the implementation with Erlang :queue the performance can be improved by about 100 times.
I ran a quick test with benchfella and got this result:

benchmark name iterations average time
Comparison stack 5000 648.01 µs/op
Erlang queue 1000 2301.37 µs/op
Que queue 10 188906.30 µs/op
Improved Que queue 500 3484.63 µs/op

Edit: This is the bench script