Add support for fast fail when token bucket empty
jamesls opened this issue · comments
Describe the feature
In adaptive mode, acquiring a send token from the token bucket always blocks:
botocore/botocore/retries/adaptive.py
Line 59 in 70514ef
botocore/botocore/retries/bucket.py
Line 66 in 70514ef
We should also support the fast fail mode (block=False
) where we propagate a client exception when the token bucket is empty.
Use Case
This is required behavior for the SDKs to allow the option to fast fail when the token bucket is empty.
The use case here is that propagating the token bucket acquisition failure lets the caller decide what to do. They could then continue to propagate that up the stack until something has enough context to decide the best course of action. With the current behavior of blocking, the caller has to always accept a block.
A concrete scenario of this would be when botocore is being used in the context of a web service. In some cases, the desired behavior when the token bucket is empty is to send back a 429/4xx response back to the caller asking them to slow down instead of the request handler blocking until capacity becomes available.
Proposed Solution
- Add some config option in the
retries
dict of the config option. Not sure what to call it, maybefast_fail
,token_bucket_block
, etc. Open to ideas. - Plumb this through into the retry handler.
- Add a new exception when this condition is hit, maybe
class NoCapacityError(BotoCoreError)
,class TokenBucketThrottleError(BotoCoreError)
?
Other Information
No response
Acknowledgements
- I may be able to implement this feature request
- This feature might incur a breaking change
SDK version used
All
Environment details (OS name and version, etc.)
All