Sorted collections get weird performance under zipf read workload
JiayuZzz opened this issue · comments
benchmark: scripts/basic_benchmarks.py
zipf read of sorted collection is significantly slower than random read, and miss some keys.
benchmark result:
- random read
[LOG] time 0 ms: Initializing PMem size 412316860416 in file /mnt/pmem0/kvdk/data
[LOG] time 894 ms: Map pmem space done
[LOG] time 49673 ms: In restoring: iterated 846691530 records
init 0 write threads
init 64 read threads
------- ops in seconds -----------
time (ms), read ops, not found, write ops, total read, total write
1000 58199000 0 0 58199000 0
2001 58351000 0 0 116550000 0
3002 58362000 0 0 174912000 0
4003 58273000 0 0 233185000 0
5004 58140000 0 0 291325000 0
6005 58131000 0 0 349456000 0
7006 58152000 0 0 407608000 0
8007 58152000 0 0 465760000 0
9008 58140000 0 0 523900000 0
10009 58140000 0 0 582040000 0
finish bench
------------ statistics ------------
read ops 58204000, write ops 0
[LOG] time 59690 ms: Closing instance ...
[LOG] time 59691 ms: Waiting bg threads exit ...
[LOG] time 59715 ms: Instance closed
- zipf read
[LOG] time 0 ms: Initializing PMem size 412316860416 in file /mnt/pmem0/kvdk/data
[LOG] time 865 ms: Map pmem space done
[LOG] time 49674 ms: In restoring: iterated 846579819 records
init 0 write threads
init 64 read threads
------- ops in seconds -----------
time (ms), read ops, not found, write ops, total read, total write
1000 14564000 589000 0 14564000 0
2001 14692000 629000 0 29256000 0
3002 14679000 628000 0 43935000 0
4003 14692000 633000 0 58627000 0
5004 14683000 626000 0 73310000 0
6005 14642000 628000 0 87952000 0
7006 14685000 626000 0 102637000 0
8007 14694000 632000 0 117331000 0
9008 14688000 629000 0 132019000 0
10009 14668000 624000 0 146687000 0
finish bench
------------ statistics ------------
read ops 14668800, write ops 0
[LOG] time 59692 ms: Closing instance ...
[LOG] time 59692 ms: Waiting bg threads exit ...
[LOG] time 59701 ms: Instance closed
string benchmarks perform well under both read workloads