S3 Usage
vysecurity opened this issue · comments
Is it normal that the S3 usage is so high?
Yes, actually it depends on how big is your input file and how many chunks you are creating with it. more chunks = more s3.
Although you might see high number of s3 API requests, I think the cost is not that high. For reference, I have more than 300k s3 requests in the last month and the current cost for me is < 2 usd.
I'm planning to update the script to work around this, until then I would suggest setting up alerts and budgets.
What I'm saying is I've used 200k executions by running like 10 commands?
No, that doesn't sound right, it can't be that high with just 10 commands. Can you enable request metrics on your bucket and see how many requests are happening?
Go to S3 -> Your bucket -> Metrics -> View additional charts -> Request metrics and create a filter with scope "This filter applies to all objects in the bucket"