Manual cleanup command
edef1c opened this issue · comments
I'd forgotten to give deletion privileges on my Google Cloud Storage bucket to the service account that Litestream was using, since the manual only says to give Storage Object Creator and Storage Object Viewer roles to the service account. As a result, attempted deletion of the previous snapshot failed, and restarting replication didn't seem to pick up on the old snapshot still existing at all.
I ended up manually deleting the snapshot and outdated WAL segments, with object versioning turned on. I needed one more WAL segment than expected: 00000051.snapshot.lz4
depends on 00000050_00000000.wal.lz4
, and recovering that got me past the "missing initial wal segment" error.
I think having some tooling for cleaning up outdated snapshots and WAL segments would make this easier.
We have a need for (preferably) Litestream to remove everything from an S3 bucket that it may have created when a database is being sunset. It's a work-in-progress but would be interesting to hear if the semantics are at all what would be acceptable.
I've added both -all-files
and -generation
flags to select between the two and it's mostly based on exposing the existing DeleteAll
function of the S3 replica.
The delete command removes data from replicas. Either a generation or -all-files
must be specified.
Usage:
litestream delete [arguments] DB_PATH
litestream delete [arguments] REPLICA_URL
Arguments:
-config PATH
Specifies the configuration file.
Defaults to /etc/litestream.yml
-no-expand-env
Disables environment variable expansion in configuration file.
-replica NAME
Optional, filters by replica.
-generation NAME
Optional, selects a generation.
-all-files
Optional, removes everything on replica path recursively.