benbjohnson / litestream

Streaming replication for SQLite.

Home Page:https://litestream.io

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Manual cleanup command

edef1c opened this issue · comments

commented

I'd forgotten to give deletion privileges on my Google Cloud Storage bucket to the service account that Litestream was using, since the manual only says to give Storage Object Creator and Storage Object Viewer roles to the service account. As a result, attempted deletion of the previous snapshot failed, and restarting replication didn't seem to pick up on the old snapshot still existing at all.
I ended up manually deleting the snapshot and outdated WAL segments, with object versioning turned on. I needed one more WAL segment than expected: 00000051.snapshot.lz4 depends on 00000050_00000000.wal.lz4, and recovering that got me past the "missing initial wal segment" error.
I think having some tooling for cleaning up outdated snapshots and WAL segments would make this easier.

We have a need for (preferably) Litestream to remove everything from an S3 bucket that it may have created when a database is being sunset. It's a work-in-progress but would be interesting to hear if the semantics are at all what would be acceptable.

I've added both -all-files and -generation flags to select between the two and it's mostly based on exposing the existing DeleteAll function of the S3 replica.

The delete command removes data from replicas. Either a generation or -all-files
must be specified.

Usage:

        litestream delete [arguments] DB_PATH

        litestream delete [arguments] REPLICA_URL

Arguments:

        -config PATH
            Specifies the configuration file.
            Defaults to /etc/litestream.yml

        -no-expand-env
            Disables environment variable expansion in configuration file.

        -replica NAME
            Optional, filters by replica.

        -generation NAME
            Optional, selects a generation.

        -all-files
            Optional, removes everything on replica path recursively.

beeper#1