Kill task using exclusive locks currently and no way to change it
IgorBerman opened this issue · comments
Description
Currently kill tasks take exclusive locks only(hardcoded). I believe there will be benefit to permit it in some scenarios to run without lock at all or append lock(TBD)
Motivation
In concurrent append and replace lock system we can run kafka ingestors in parallel to compaction tasks. However multiple compaction will create more unused segments, so to keep it controllable it will be good to clean every segment after compaction task finished. However currently kafka ingestors lock intervals with append lock, so kill task can't cleanup unused segments.
So my proposition is to permit kill task to run with replace lock(same as of compaction task) or no lock at all if we not marking segments as used(for any reason).
This should permit kill , compaction and kafka ingestors tasks to run concurrently, thus achieving best parallelism