Allow to reset storage caches at next restart

[Nikolay]

Is your feature request related to a problem? Please describe.

Now its hard to reset internal storage caches at kubernetes based deployments. storage pods configured with the same parameters and managed by replicaset-controller. So its not possible to stop one pod, cleanup cache and start it again.

Describe the solution you'd like
Storage cache can be reset on the next service start with special flag file reset_cache_on_startup, that must be placed at cache/ directory. After cache reset, storage service must remove this flag from directory.

vmsingle usage:

kubectl exec -it vmsingle-pod sh
touch victoria-metrics-data/cache/reset_cache_on_startup
kubectl delete pod vmsingle-pod

vmstorage usage:

kubectl exec -it vmstorage-pod-0
touch vmstorage-data/cache/reset_cache_on_startup
kubectl delete pod vmstorage-pod-0

Describe alternatives you've considered
remove cache from disk and perform kill -9, it may lead to dataloss for storage.

[Aliaksandr Valialkin]

VictoriaMetrics should automatically remove the <-storageDataPath>/cache directory on startup if the directory contains reset_cache_on_startup file starting from the commit 682662b .

[Aliaksandr Valialkin]

VictoriaMetrics should automatically remove the <-storageDataPath>/cache directory on startup if the directory contains reset_cache_on_startup file starting from v1.63.0. Closing the feature request as done.