keredson / shadb

SHADB

This is a git-backed document store with indexing.

Usage

Load your database and define your indexes in python.

db = shadb.SHADB('/some/directory')
db.add_index('by_id', lambda o: o.get('id'), unique=True)
db.add_index('by_type', lambda o: o.get('type'))

Store a document. If a document doesn't have an id, one will be generated.

>>> alice = {'name':'Alice', 'type':'user'}
>>> bob = {'name':'Bob', 'type':'user'}
>>> db.store(alice, bob)
>>> print(alice)
{'name':'Alice', 'type':'user', 'id':'mySSkm8obM6m6MRzYJoaBH'}

Retrieve a document:

>>> db.docs.by_id['mySSkm8obM6m6MRzYJoaBH']
{'name':'Alice', 'type':'user', 'id':'mySSkm8obM6m6MRzYJoaBH'}
>>> db.docs.by_type['user']
[
  {'name':'Alice', 'type':'user', 'id':'mySSkm8obM6m6MRzYJoaBH'},
  {'name':'Bob', 'type':'user', 'id':'8Ry2m7WGoEZEJE3y4hTwsq'}
]

Indexes support SQLite's like syntax:

>>> db.add_index('by_name', lambda o: o.get('name'))
>>> db.docs.by_name.keys(like='al%')
['Alice',]
>>> db.docs.by_name.items(like='al%')
[
  ('Alice', [{'name':'Alice', 'type':'user', 'id':'mySSkm8obM6m6MRzYJoaBH'}]),
]
>>> db.docs.by_name.values(like='al%')
[{'name':'Alice', 'type':'user', 'id':'mySSkm8obM6m6MRzYJoaBH'}]

A document's filename can be retrieved by the index. Filenames are automatically generated by the object's id and (optional) type.

>>> db.idx.by_id['mySSkm8obM6m6MRzYJoaBH']
'user/m/y/S/S/user-mySSkm8obM6m6MRzYJoaBH.json'

And loaded by filename:

>>> db.load('user/m/y/S/S/user-mySSkm8obM6m6MRzYJoaBH.json')
{'name':'Alice', 'type':'user', 'id':'mySSkm8obM6m6MRzYJoaBH'}

Commits

Objects are stored as JSON files in a git repository. To commit, use with:

with db:
  db.store({'name':'Alice', 'type':'user'})

To specify a commit message:

with db.commit(m='a commit message') as commit:
  commit.store({'name':'Alice', 'type':'user'})

To commit outside a context manager:

fns = db.store({'name':'Alice', 'type':'user'}, {'name':'Bob', 'type':'user'})
db.commit(*fns, m='a commit message')

Object Types

Stored objects can be anything JSON serializable, plus dataclass and namedtuple. Example:

@dataclass
class User:
   id: int
   name: str

>>> db = shadb.SHADB('/some/directory')
>>> db.register(User)
>>> db.store(User(1, 'Alice'))
'User/1/User-1.json'
>>> db.load(fn)
User(id=1, name='Alice')

Install

$ pip install shadb