Giters

redis / redis-benchmarks-specification

The Redis benchmarks specification describes the cross-language/tools requirements and expectations to foster performance and observability standards around redis related technologies. Members from both industry and academia, including organizations and individuals are encouraged to contribute.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

[BUG]: Failures in benchmarking tests that use dataset configured in dbconfig

odidev opened this issue · comments

commented

Describe the bug

I am executing benchmarking tests on the Linux/AMD64 AWS instance.

I am facing issues in all the tests that have dataset configured in dbconfig. For example, consider the test “memtier_benchmark-1key-geo-60M-elements-geohash-pipeline-10.yml”. I ran the test with dataset, following the command below:

redis-benchmarks-spec-client-runner --db_server_host localhost --db_server_port 6379 --client_aggregated_results_folder ./test --flushall_on_every_test_start --run-tests-with-dataset –test memtier_benchmark-1key-geo-60M-elements-geohash-pipeline-10.yml

The failure logs are as below:

2023-06-27 10:43:03 ERROR The total numbers of keys in setup does not match the expected spec: 1!=0. Aborting... 
2023-06-27 10:43:03 CRITICAL Some unexpected exception was caught during local work. Failing test....

I found a similar issue here: < #206 >, and the solution was this PR: < https://github.com/redis/redis-benchmarks-specification/pull/213/files >.

Following the above PR, I edited the “keyspacelen” from 1 to 0, and the test passes successfully.

Similarly, there are 8 more tests in the suite, that use dataset. Below is the list:

memtier_benchmark-1key-geo-60M-elements-geohash.yml
memtier_benchmark-1key-geo-60M-elements-geopos-pipeline-10.yml
memtier_benchmark-1key-geo-60M-elements-geopos.yml
memtier_benchmark-1key-geo-60M-elements-geosearch-fromlonlat-bybox.yml
memtier_benchmark-1key-geo-60M-elements-geodist-pipeline-10.yml
memtier_benchmark-1key-geo-60M-elements-geosearch-fromlonlat-pipeline-10.yml
memtier_benchmark-1key-geo-60M-elements-geodist.yml
memtier_benchmark-1key-geo-60M-elements-geosearch-fromlonlat.yml

All the above-mentioned 9 tests use dataset and passes successfully after editing “keyspacelen” from 1 to 0.

Do you think it is reasonable to edit the “keyspacelen” to 0? If yes, I will raise a PR for the same.

Kindly let me know your thoughts on the above.

Environment (please complete the following information):

  • OS: Ubuntu 22.04
  • PyPi version package pip 23.1.2 from /usr/local/lib/python3.10/dist-packages/pip (python 3.10)
  • Platfrom x86_64
commented

Hi team, a gentle reminder!!
Can you please look into this issue?

commented

Hi Team, a gentle reminder!!
Can you please look into this ticket?