[Feature] Limit per-user requests occupied bandwidth
Smelly-calf opened this issue · comments
Is your feature request related to a problem? Please describe.
Limit the bandwidth occupied by user requests to reduce the impact of importing large offline data queries on other queries.
Describe the solution you'd like
Add user packet size configs:
packet_size_token_limit_burst, which is the burst of the request packet size token bucketpacket_size_token_rate, which is the token produced rate
Describe alternatives you've considered
no alternative
Additional context
why would chproxy could harm your DNS? Its not obvious because chproxy is not supposed to trigger a lot of http calls to clickhouse. Indeed, clickhouse would be KO way before your DNS (otherwise you have an issue with your DNS).
Moreover, I don't understand why your fix solve your issue. The amount of bytes transferred is not related to the usage of the DNS. Unless I'm mistaken it's the amount of HTTP calls that impacts the usage of your DNS (and normally you have an local cache that prevents the server fetching the DNS for a domain already called). And you can already limit the number of http calls using the max_queue_size & max_concurrent_queries parameters
why would chproxy could harm your DNS? Its not obvious because chproxy is not supposed to trigger a lot of http calls to clickhouse. Indeed, clickhouse would be KO way before your DNS (otherwise you have an issue with your DNS).
Moreover, I don't understand why your fix solve your issue. The amount of bytes transferred is not related to the usage of the DNS. Unless I'm mistaken it's the amount of HTTP calls that impacts the usage of your DNS (and normally you have an local cache that prevents the server fetching the DNS for a domain already called). And you can already limit the number of http calls using the max_queue_size & max_concurrent_queries parameters
yes, it does't matter with DNS, and i've changed the description. We try to implement bandwidth limit of user requests, which aiming to reduce the impact of importing large number of offline data queries on other queries.
thanks, it's clearer now
since you're PR was merged, I'm closing this issue.
We will try to release a new version of chproxy next week