renproject / kv

A flexible and extensible library for key-value storage

📦 kv

A flexible and extensible library for key-value storage.

• Multiple encoding/decoding formats
• Persistent database drivers
• In-memory database drivers
• Time-to-live caching
• Safe for concurrent use
• Production ready

Installation

go get github.com/renproject/kv

Requirements

Requires go1.11 or newer.

Usage

Codec

A Codec encodes interface{} values into bytes, decode bytes into the interface{} values. Generally, when a specific type is not supported, a Codec will panic. Out of the box, KV supports:

• JSONCodec which encodes/decodes using the standard library JSON marshaler, and
• GobCodec which encodes/decodes using the standard library [Gob]https://golang.org/pkg/encoding/gob marshaler (you must explicitly register types outside of KV).

An example of using the JSONCodec:

db := kv.NewLevelDB(".db", kv.JSONCodec)

DB

A DB is a key/value database. The key is a string and the value is an interface{} that can be encoded/decoded by the chosen Codec (different DBs can use different Codecs). A DB is safe for concurrent if, and only if, the underlying driver is safe for concurrent use (the LevelDB driver and BadgerDB driver are safe for concurrent use).

An example of initialising a DB:

// Initialising an in-memory database 
db := kv.NewMemDB(kv.JSONCodec)

// Initialising a LevelDB database
db = kv.NewLevelDB(".ldb", kv.JSONCodec)

// Initialising a BadgerDB database 
db = kv.NewBadgerDB(".bdb", kv.JSONCodec)

Although reading/writing is usually done through a Table, you can read/write using the DB directly (you must be careful that keys will not conflict with Table name hashes):

// Write
if err := db.Insert("key", "value"); err != nil {
    log.Fatalf("error inserting: %v", err)
}

// Read
var value string
if err := db.Get("key", &value); err != nil {
    log.Fatalf("error getting: %v", err)
}

// Delete
if err := db.Delete("key"); err != nil {
    log.Fatalf("error deleting: %v", err)
}

// Number of key/value pairs with the given prefix
size, err := db.Size("")
if err != nil {
    log.Fatalf("error sizing: %v", err)
}
log.Printf("%v key/value pairs found", size)

// Get an iterator over all key/value pairs with the given prefix
iter := db.Iterator("")

Table

A Table is an abstraction over a DB partitions key/value pairs into non-overlapping groups. This allows you to iterate over small groups of key/value pairs that are logically related. You must ensure that Table names are unique.

An example of basic use:

type Foo struct{
    A string
    B int
    C []byte
}

// Init
table := kv.NewTable(db, "myAwesomeTable")

// Write
foo := Foo{"foo", 420, []byte{1,2,3}}
if err := table.Insert("key", foo); err != nil {
    log.Fatalf("error inserting into table: %v", err)
}

// Read
bar := Foo{}
if err := table.Get("key", &bar); err != nil {
    log.Fatalf("error getting from table: %v", err)
}

The most useful feature of Tables is iteration:

// Get the number of key/value pairs in the table
size, err := table.Size()
if err != nil {
    log.Fatalf("error sizing table: %v", err)
}
log.Printf("%v key/value pairs found", size)

// Iterate over all key/value pairs in the table
for iter := table.Iterator(); iter.Next(); {
    key, err := iter.Key()
    if err != nil {
        continue
    }
    value := Foo{}
    if err = iter.Value(&value); err != nil {
        continue
    }
}

Benchmarks

Database	Number of iterations run	Time (ns/op)	Memory (bytes/op)
LevelDB	2000	10784337	4397224
BadgerDB	100	200012411	200012411

Contributors

Built with ❤ by Ren.