simerplaha / SwayDB

Persistent and in-memory key-value storage engine for JVM that scales on a single machine.

Home Page:https://swaydb.simer.au

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

SwayDB Gitter Chat Build status Maven central

Persistent and in-memory key-value storage engine for the JVM aimed at high performance & resource efficiency.

Small footprint: around 7.1 MB jar size. No external core dependency (#307).

Scalable on a single machine: Distribute data on multiple local SSDs. Allocate single or multiple Threads for reads, caching & compaction.

Branches: This is the master branch. See develop for latest (unstable).

Status: Under testing & performance optimisations. See project status.

Documentation

Sponsors

Thank you JetBrains & JProfiler for full-featured open-source licences to their awesome development tools!

Jetbrains support JProfiler support [Become a sponsor]

Overview

  • Simple data types - Map, Set, Queue, SetMap & MultiMap with native Java and Scala collections support.
  • Conditional updates using any pure JVM function - No query language.
  • Atomic updates and inserts with Transaction API.
  • Non-blocking core with configurable APIs for blocking, non-blocking and/or reactive use-cases.
  • Single or multiple disks persistent, in-memory or eventually persistent.
  • Streams - Async & sync forward and reverse data streaming/iteration.
  • TTL - non-blocking, non-polling asynchronous auto expiring key-values.
  • Range operations to update, remove & expire a range of key-values.
  • Key only iterations (Lazily fetched values).
  • Configurable compression with LZ4 & Snappy
  • Configurable core internals to support custom workloads.
  • Duplicate values elimination with compressDuplicateValues.

Use cases

Highly configurable to suit different workloads. Some known use-cases are:

  • General key-value storage
  • Message queues
  • Time-series or Events data
  • Caching
  • Application logs
  • Archiving data or cold storage with high file level compression

Quick start

Contributing

Contributions are encouraged and welcomed. We are here to help and answer any questions.

Code of conduct - Be nice, welcoming, friendly & supportive of each other. Follow the Apache foundation's COC.

  • Contributing to data management API

    • Build new data structures extending existing data structures. See MultiMap for example which is an extension of Map.
      • Graph
      • List
      • Geospatial
      • Logs
      • Observables
      • Full-text search
      • etc
    • Add support for external serialising, streaming & compression libraries.
    • Test and improve existing data structures.
  • Contributing to core API

  • Contributing to core internals

    • See code marked TODO.
    • Pick any small section to improve. You will find that everything is a function and can be unit-tested independently, so you can easily pick anything to work on.

Project status

Under testing & performance optimisations.

Your feedback and support is very important to get to production. Please get involved via chat, discussion, issues or by becoming a sponsor.

Future releases might not be backward compatible until we are production ready.

See tasks labelled Production release that are required before becoming production ready.

Related GitHub projects

About

Persistent and in-memory key-value storage engine for JVM that scales on a single machine.

https://swaydb.simer.au

License:Apache License 2.0


Languages

Language:Scala 97.9%Language:Java 2.1%