Please consider using GetParameter(s) instead of GetParametersByPath
nachmore opened this issue · comments
We have encountered cases where deploying chamber at scale runs into AWS API limits.
GetParametersByPath
has a maximum tps of 100
with customers able to increase this slightly by opening a support case with AWS. GetParameter
and GetParameters
has a maximum tps of 3,000
and is more suited for high throughput use cases (ref).
In general some kind of caching layer reducing the calls to GetParametersByPath
might be useful to allow chamber to be deployed at scale - and will likely improve overall performance.
Chamber uses GetParametersByPath
to find all secrets (parameters) for a given service. It's used primarily for loading secrets into environment variables for ensuing commands, but it's also used for export. Based on git history, DescribeParameters
used to be employed but was swapped out for GetParametersByPath
to take advantage of higher API limits (ironically). The current limit for GetParametersByPath
is indeed 100 tps, but I'm not sure what the limit is for DescribeParameters
. Based on this SSM doc it might be 40.
I looked at GetParameters
, and it requires a list of ARNs for the parameters to get. Unless it allows wildcards - I tried and can't get them to work - then it won't work as a substitute, because chamber doesn't already know the ARNs of the secrets it's looking for.
Caching makes sense, but it may be better to have that layer outside chamber. Another tactic is to set retries and (coming up soon) using the adaptive retry mode for AWS calls; either of those may cope with rate limits to an extent.
Thanks for the follow up! The limits for DescribeParameters
are very low - I'll follow up to see if those can be documented publicly. Is there an existing pattern of caching external data (like this) outside of Chamber?