Is there a bulk Deletion API somewhere?

[Thomas Ganter]

Is your feature request related to a problem? Please describe.
I manage to ingest my logs and telemetry from multiple hosts and in various formats fine. But now that I collect Gigabytes of data every day, I look towards housekeeping and how to remove certain entries from the store, e. g. that have only relevance during a very brief immediate time window.

Describe the solution you'd like
There is a bulk deletion API on elastic, that allows to delete documents that match a query.
Something comparable would be nice.

Describe alternatives you've considered
I am currently experimenting with a python script that selects documents and then idividually issues single document deletion requests. That works, but I wonder whether there is a better option.

Additional context
-none-

[Prabhat Sharma]

Currently we don't have a bulk deletion API. In general an approach to have only the latest data and conserve space would be to have your indexes automatically created based on date. See the picture below. Configuration for fluent bit for this in in Zinc documentation.

Once this is implemented, you can delete the entire index instead if deleting the records individually. We do have an API available to delete the entire index.

Ideally speaking once we move the s3 storage from experimental to alpha/beta you can default to storing index data to s3 where storage cost is much lower and you don't have to worry about disk space management.