Val9 / livedb

A database frontend for concurrent editing systems

LIVE DB!

This is a database wrapper which exposes the API that realtime databases should have.

You can submit operations (edit documents) and subscribe to documents. Subscribing gives you a stream of all operations applied to theh given document. You can also make queries, which give you a feed of changes in the result set while the query is open.

Currently this is very new and only used by ShareJS. To use it, you need a snapshot database wrapper. The obvious choice is mongodb. A database wrapper for mongo is available in share/livedb-mongo.

Data Model

In LiveDB's view of the world, every document has 3 properties:

• version: an incrementing number starting at 0
• type: an OT type. OT types are defined in share/ottypes. Types are referenced using their URIs (even though those URIs don't actually mean anything). Documents which don't exist implicitly have a type of null.
• data: The actual data that the document contains. This must be pure acyclic JSON. Its also type-specific. (JSON type uses raw JSON, text documents use a string, etc).

LiveDB implicitly has a record for every document you can access. New documents have version 0, a null type and no data. To use a document, you must first submit a create operation, which will set the document's type and give it initial data. Then you can submit editing operations on the document (using OT). Finally you can delete the document with a delete operation. By default, livedb stores all operations forever - nothing is truly deleted.

Using Livedb

Livedb requires a backend database to store snapshots & operations. You can put snapshots & operations in different places if you want, though its easier to put all data in the same place.

The backend database(s) needs to implement a simple API which has documentation and a sample implementation here. Currently the only database binding is livedb-mongo.

A livedb client is created using either an options object or a database backend. If you specify a database backend, its used as both oplog and snapshot.

db = require('livedb-mongo')('localhost:27017/test?auto_reconnect', {safe:true});
livedb = require('livedb').client(db);

Or using an options object:

db = require('livedb-mongo')('localhost:27017/test?auto_reconnect', {safe:true});
livedb = require('livedb').client({db:db});

You can use a different database for both snapshots and operations:

snapshotdb = require('livedb-mongo')('localhost:27017/test?auto_reconnect', {safe:true});
oplog = {writeOp:..., getVersion:..., getOps:...};
livedb = require('livedb').client({snapshotDb:snapshotdb, oplog:oplog});

client({db:db}); is a shorthand for client({snapshotDb:db, oplog:db}).

The options object can also be passed:

• redis: redis client. This can be specified if there is any further configuration of redis that you want to perform. The obvious examples of this are when redis is running on a remote machine, redis requires authentication or you want to use something other than redis db 0.
• redisObserver: redis client. Livedb actually needs 2 redis connections, because redis doesn't let you use a connection with pubsub subscriptions to edit data. Livedb will automatically try to clone the first connection to make the observer connection, but we can't copy some options. if you want to do anything thats particularly fancy, you should make 2 redis instances and provide livedb with both of them. Note that because redis pubsub messages aren't constrained to the selected database, the redisObserver doesn't need to select the db you have your data in.
• extraDbs: {name:query db} This is used to register extra database backends which will be notified whenever operations are submitted. They can also be used in queries.

Creating documents

To create a document, you first need to submit a create operation to the document to set its type. A document doesn't properly exist until it has a type --- it certainly can't store any data.

A create operation looks like this: {create:{type:TYPE, [data:INITIAL DATA]}, [v:VERSION]}. The type should be something accessible in the map returned by require('ottypes'), for example json0 or http://sharejs.org/types/textv1. Specifying initial data is optional. If provided, it is passed to the type's create() method. This does what you expect - for JSON documents, pass your initial object here. For text documents, pass a string containing the document's contents. As with all operations, the version is optional. You probably don't want to specify the version for a create message.

For example:

livedb.submit('users', 'fred', {create:{type:'json0', data:[1,2,3]}}, function(err, version, transformedByOps, snapshot) {
  // I made a document, ma!
});

Since documents implicitly exist with no type at version 0, usually the create message will increment the version from 0 to 1. Not all documents you want to delete have a version of 0 - if a document is deleted, it will retain its version.

Deleting documents

Deleting documents is similar to creating them. A deleted document has no type and no data, but will retain its version (actually, the delete operation will bump the document's version). A delete operation looks like this: {del:true, [v:VERSION]}.

livedb.submit('users', 'fred', {del:true}, function(err) {
  //goneskies! Kapow!
});

Editing documents

You edit a document by submitting an operation. Operations are OT type-specific JSON blobs. Refer to the documentation on the particular OT type for details. For example, text documents are documented here. If we had a text document stored in LiveDB and wanted to edit it, it might look like this:

livedb.submit('love letters', 'dear fred', {op:[6, "You never return my calls!"], v:1002}, function(err) {
  // ...
});

You should always specify the version when submitting operations. If you don't, operations will do funny things in the face of concurrency.