There is big difference between the results of multiple list tests

[sirizhou]

I use following script to benchmark list operation

#!/bin/bash
for loop in 1 2 3 4 5
do 
echo "loop value is : $loop"
echo 3 > /proc/sys/vm/drop_caches
echo put...
warp put --obj.size=128kb --concurrent=20 --host=127.0.0.1:9000 --access-key=minioadmin --secret-key=minioadmin --duration=10m --bucket=put-benchmark-bucket
warp get --objects=1000000 --obj.size=128kb --concurrent=20 --host=127.0.0.1:9000 --access-key=minioadmin --secret-key=minioadmin --duration=10m --bucket=get-benchmark-bucket
warp list --objects=1000000 --obj.size=128kb --concurrent=20 --host=127.0.0.1:9000 --access-key=minioadmin --secret-key=minioadmin --duration=10m --bucket=list-benchmark-bucket
sleep 2m
done

The results are as follow

TLC

  | put | get | list
  | put(MiB/s) | put(MiB/s) | get(Mib/s) | put(Mib/s) | list(obj/s)
1st | 422.11 | 444.34 | 1406.51 | 452.96 | too few samples
2nd | 472.59 | 442.28 | 1421.12 | 463.75 | 114005.01
3rd | 465.25 | 463.77 | 1432.41 | 471.46 | 128908.69
4th | 483.33 | 462.35 | 1439.18 | 471.02 | 208446.33
5th | 488.03 | 475.03 | 1441.88 | 468.91 | 138895.01

as you can see, the list results vary for each other. the 4th list speed is twice than the 1st.
the backend storage is two tlc ssds which is divided into two patrition.

if i use Optane SSD as back end storage. results are as follows:

Optane

  | put | get | list
  | put(MiB/s) | put(MiB/s) | get(Mib/s) | put(Mib/s) | list(obj/s)
1st | 532.57 | 515.52 | 1726.61 | 508.62 | 106094.92
2nd | 525.09 | 538.18 | 1825.02 | 520.19 | 234757.65
3rd | 532.93 | 532.01 | 1849.24 | 535.71 | 230527.95
4th | 533.47 | 538.77 | 1856.61 | 540.65 | 113711.63
5th | 542.47 | 544.16 | 1858.99 | 538.82 | too few samples

commit d3ee60c (tag: v0.3.40)

[Klaus Post]

You are likely just seeing server variance.

If the servers cycle the bloom filter between put and list it will recognize that no updates have taken place and will be able to serve directly from cache.

@sirizhou If you attach the zst files (they shouldn't contain sensitive information) I can check if everything looks correct.

[Klaus Post]

Closing since no response.