Add information about partition limit on write
sanderson opened this issue · comments
InfluxDB Cloud Serverless limits the number of partitions per database that can be written to in ~15 minute periods to ensure availability and consistent service for other tenants. Serverless partitions data by day, so the limit will be hit when a user tries to write data with timestamps from multiple days within a 15 minute period.
We need to add this limit to the Global limits section in the docs.
@domodwyer What is the specific number of partitions that we limit to?
Relevant URLs
Err 5,000 - so that works out to be more than 13 years of data in 15m!
Note it only creates partitions / counts towards the 15m limit when a user writes to a date, not the interval, so writing a value for monday and wednesday doesn't turn into [monday, tuesday, wednesday]
, it's just [monday, wednesday]
.
This must be 5,000 days of writes!
Thanks for this @sanderson 🙇
Thanks @domodwyer! Question:
Note it only creates partitions / counts towards the 15m limit when a user writes to a date, not the interval, so writing a value for monday and wednesday doesn't turn into [monday, tuesday, wednesday], it's just [monday, wednesday].
Is the limit on the number of new partitions created, or the number of distinct partitions (existing or new) that data is written to?
Distinct, in a 15m window.
You can write to 5,000 partitions, wait 20 minutes, and then write to those 5,000 partitions again, but you can't wait and the write to 5,001 - at any one time, the system will allow at most 5,000 non-empty partitions in a 15m period.
Note the period can be changed at a service / platform level for various operational reasons, but it's been 15m for a long time historically speaking.