BlobsFile is an append-only (i.e. no update and no delete) content-addressed blob store (using BLAKE2b as hash function).
It draws inspiration from Facebook's Haystack, blobs are stored in flat files (called BlobFile) and indexed by a small kv database for fast lookup.
BlobsFile is BlobStash's storage engine.
- Durable (data is fsynced before returning)
- Immutable (append-only, can't mutate or delete blobs)
- Optional compression (Snappy or Zstandard)
- Extra parity data is added to each BlobFile (using Reed-Solomon error correcting code), allowing the database to repair itself in case of corruption.
- The test suite is literraly punching holes at random places