Can the workload rate be customized?

Question

Can the workload rate be customized?

WangTingZheng opened this issue a year ago · comments

For example, we can simulate real-world workloads by customizing the number of load arrivals for each time period like this:

Youngjae Lee · Answer 1 · Tue Jun 13 2023 20:30:50 GMT+0800 (China Standard Time)

Since YCSB is usually used to measure peak throughput, it does not have a request rate customization.
I think we could add some simple constant rate limit algorithms, but simulating a workload like the graph above seems complicated. Is there any popular workload trace available that can be used as an input?

王听正 · Answer 2 · Sat Jun 17 2023 12:16:53 GMT+0800 (China Standard Time)

Since YCSB is usually used to measure peak throughput, it does not have a request rate customization. I think we could add some simple constant rate limit algorithms, but simulating a workload like the graph above seems complicated. Is there any popular workload trace available that can be used as an input?

Thank you for your reply，QueryBot5000[1] has collected the number of accesses within certain intervals in certain scenarios, which might be useful?

[1] Ma, Lin, et al. "Query-based workload forecasting for self-driving database management systems." Proceedings of the 2018 International Conference on Management of Data. 2018.

Youngjae Lee · Answer 3 · Mon Jul 03 2023 15:08:35 GMT+0800 (China Standard Time)

Hi, sorry for the late reply.

I've added a way to limit the client request rate.
Check out the rate_limit branch.

A constant rate can be specified using the limit.ops option. (initial rate)

The rate can be also set from the trace file by using limit.file options. The format of the file is as follows.

An example command using the above two options would be:

./ycsb -run -db basic -P workloads/workloada  -p threadcount=4 -p basic.silent=true -p limit.ops=100 -p limit.file=rate_trace.txt -s

Rate limiting only works for the run phase.

Please let me know if you have any issues or improvement ideas :)

王听正 · Answer 4 · Mon Jul 17 2023 13:08:42 GMT+0800 (China Standard Time)

Hi, sorry for the late reply.

I've added a way to limit the client request rate. Check out the rate_limit branch.

A constant rate can be specified using the limit.ops option. (initial rate)

The rate can be also set from the trace file by using limit.file options. The format of the file is as follows.
20 288
30 292
40 368
50 335
60 242
An example command using the above two options would be:
./ycsb -run -db basic -P workloads/workloada  -p threadcount=4 -p basic.silent=true -p limit.ops=100 -p limit.file=rate_trace.txt -s
Rate limiting only works for the run phase.

Please let me know if you have any issues or improvement ideas :)

thanks a lot!

王听正 · Answer 5 · Sat Mar 09 2024 18:35:30 GMT+0800 (China Standard Time)

how to get read throughput and latency when I set the rate limite? throughput and latency is fixed. @ls4154

Youngjae Lee · Answer 6 · Sat Jun 01 2024 14:59:53 GMT+0800 (China Standard Time)

how to get read throughput and latency when I set the rate limite? throughput and latency is fixed.

If you limit the maximum rate, the rate should be fixed, unless your db is slower than limits.
Latency should not be fixed though.