sled
An (alpha) modern embedded database.
use sled::Tree;
let tree = Tree::start_default(path)?;
// set and get
tree.set(k, v1);
assert_eq!(tree.get(&k), Ok(Some(v1)));
// compare and swap
tree.cas(k, Some(v1), Some(v2));
// scan forward
let mut iter = tree.scan(k);
assert_eq!(iter.next(), Some(Ok((k, v2))));
assert_eq!(iter.next(), None);
// deletion
tree.del(&k);
// block until all operations are on-disk
tree.flush();We also support merge operators!
features
- API similar to a threadsafe
BTreeMap<Vec<u8>, Vec<u8>> - fully atomic single-key operations, supports CAS
- zero-copy reads
- subscription/watch semantics on key prefixes
- merge operators
- forward and reverse iterators
- a crash-safe monotonic ID generator capable of generating 75-125 million ID's per second
- zstd compression (use the
compressionbuild feature) - cpu-scalable lock-free implementation
- SSD-optimized log-structured storage
goals
- don't make the user think. the interface should be obvious.
- don't surprise users with performance traps.
- don't wake up operators. bring reliability techniques from academia into real-world practice.
- don't use so much electricity. our data structures should play to modern hardware's strengths.
plans
- LSM tree-like write performance with traditional B+ tree-like read performance
- MVCC, serializable transactions, and snapshots
- forward-compatible binary format
- FIFO prefix subscription semantics
- concurrent snapshot delta generation and recovery
- first-class access to replication stream
- consensus protocol for PC/EC systems
- pluggable conflict detection and resolution strategies for PA/EL systems
- SQL support
- native bindings
known issues, warnings
- the on-disk format is going to change in non-forward compatible ways
before the
1.0.0release! after that, we will always support forward migrations. - quite young, should be considered unstable for the time being
- the C API is likely to change rapidly
- writepath is not well optimized yet. readpath is essentially wait-free and zero-copy.
- 32 bit architectures require Rust nightly with the
nightlyfeature enabled.
contribution welcome!
want to help advance the state of the art in open source embedded databases? check out CONTRIBUTING.md!
architecture
lock-free tree on a lock-free pagecache on a lock-free log. the pagecache scatters partial page fragments across the log, rather than rewriting entire pages at a time as B+ trees for spinning disks historically have. on page reads, we concurrently scatter-gather reads across the log to materialize the page from its fragments. check out the architectural outlook for a more detailed overview of where we're at and where we see things going!