Changes made by Easton Lee

GroundDB 1.0 resides in master branch, supports "Resume of changes in collections and Resume of methods", is suitable for building offline-first apps.

GroundDB 2.0 resides in grounddb-caching-2016 branch, doesn't support "Resume of changes in collections and Resume of methods", supports "indexeddb and websql".

Easton modified 1.0 and fixed several bugs.

Pitfalls

1. You should adapt methodResume parameters manually.

You should add methods which you hope to resume after network recovery from offline, the default methodResume uses collections' name, like this /Lists/insert, source code is like this

  // This is how grounddb uses this internally
  Ground.methodResume([
    '/' + self.name + '/insert',
    '/' + self.name + '/remove',
    '/' + self.name + '/update'
  ], self.connection);

but the real world Meteor methods you want to resume are like this list/insert so you need to adapt the code manually.

How to find the outstanding Method methods? Read the source code in master branch groundDB.client.js/_getMethodsList, you can find clue from method._message.method of connection._methodInvokers.

2. First outstanding method is not saved.

When you are offline, and changed some data locally, your client will invoke a Meteor method and try to sync the change with the server, of course the sync will fail, then the method will be kept in connection._methodInvokers which means this method is waiting to be finished, Gound:DB will save these outstanding methods to Local Storage, but it saves too fast even before connection._methodInvokers contains the first outstanding method. So the solution is to add a delay before saving outstanding methods, that's been done in my commit.

Below is the original author @raix 's README.

ground:db

GroundDB is a fast and thin layer providing Meteor offline database and methods - Taking cloud data to the ground.

  // Return a grounded Meteor.Collection
  var list = new Ground.Collection('list');

Meteor Collection Interface

GroundDB is like a normal Meteor.Collection - but changes and outstanding methods are cached and resumed. Turn off resume by setting the option resume: false

Live basic debug test

Angular example by quanganh206

Features:

Light footprint
Broad browser support Chrome, Safari, Firefox and Internet Explorer 9
Fallback to normal Meteor.Collection if no local storage
Resume of changes in collections
Resume of methods
Works offline updating cross window tabs
Support for SmartCollection
Support for offline client-side only databases
Uses EJSON.minify and EJSON.maxify to compress data in localstorage
In the future there will be a customizable conflict handler on the server-side

Current development

Current development is the Meteor 1.2 ES2015 localforage version ground:db@1.0.0-alpha.x branch is tracked in #118 It supports nearly all types of storage even sqlite on cordova and will have better performance and general resumeability - that said it's also an api breaking version.

Creating a Ground.Collection object (variants)

  // Return a grounded Meteor.Collection
  var list = new Ground.Collection('list');

  or

  // Get the groundDB of existing Meteor.Collection
  var list = new Meteor.Collection('list');
  var groundList = new Ground.Collection(list);

  or

  // Ground an existing Meteor.Collection  
  var list = new Meteor.Collection('list');
  // just ground the database:
  Ground.Collection(list);

Example of different patterns. Grounding a Meteor.Collection will attach the cache, resume and cross tabs update offline

Pure client-side offline databases (variants)

Ground.Collection can be applied on client-side only eg.: new Meteor.Collection(null);

  // Creates client-side only database, this one maps on suffix `null`
  var list = new Ground.Collection(null);

  // Creates client-side only database, this one maps on suffix `list` *(Meteor 0.6.5+)*
  var list = new Ground.Collection('list', { connection: null });
  
  or

  // Get the groundDB of existing Meteor.Collection
  var list = new Meteor.Collection(null);
  var groundList = new Ground.Collection(list, 'list');

  or

  // Ground an existing Meteor.Collection  
  var list = new Meteor.Collection(null);
  // just ground the database and map on suffix `list`
  Ground.Collection(list, 'list');

You can only have one grounded collection with name null

Support

Tested on Chrome, Safari, Firefox and IE9 (though appcache is not supported in IE9 tabs are updated when offline) - but all browsers that support localstorage contains a FF safe test of localstorage

If localstorage is not supported the groundDB simply work as a normal Meteor.Collection

Concept

Localstorage is simple and widely supported - but slow - Thats why we only use it for caching databases and methods + trying to limit the read and writes from it.

GroundDB saves outstanding methods and minimongo db into localstorage - The number of saves to localstorage is minimized. Use Ground.resumeMethods

When the app loads GroundDB resumes methods and database changes - made when offline and browser closed.

Ground user details

It's possible to mount an allready existing collection on a groundDB eg.:

  Ground.Collection(Meteor.users);

The example will keep Meteor.user() returning correct user details - even if offline

Ground SmartCollections

It's possible to ground an allready existing smartCollectin on a groundDB eg.:

  var mySmartCollection = new SmartCollection('foo');
  Ground.Collection(mySmartCollection);

or

  var mySmartCollection = Ground.Collection(new SmartCollection('foo'));

  // use the smart collection
  mySmartCollection.insert(/* stuff */);

Resume of outstanding methods

Database changes and methods will be sent to the server just like normal. The methods are sent to server after relogin - this way this.userId is set when running on the server. In other words: Just like normal

Publish and subscription

Online

Subscription behavior when using GroundDB - When online it's just like normal Meteor so nothing new. If you unsubscribe a collection you can still insert etc. but the data will not be visible on the client.

Offline

When offline the data remains in the local database - since the publish is a server thing. Use the query selector for filtering unwanted data. When reconnected the database will update client subscription and changes will be resumed

Events - client-side

The event api is as follows:

Ground.lookup = function(collectionName) {};
Ground.methodResume = function(names, connection) {};

Ground.addListener // Listen to general events
foo.addListener // Add listener specific to the foo collection

// Reactive status of all subscriptions, ready or not:
Ground.ready();

DEPRECATED API: ~~Ground.onQuotaExceeded = function() {};~~ ~~Ground.onResumeDatabase = function(name) {};~~ ~~Ground.onResumeMethods = function() {};~~ ~~Ground.onMethodCall = function(methodCall) {};~~ ~~Ground.onCacheDatabase = function(name) {};~~ ~~Ground.onCacheMethods = function() {};~~ ~~Ground.onTabSync = function(key) {};~~ ~~Ground.skipMethods = function(methodsToSkipObject)~~

Cache methods

Use the Ground.methodResume to cache method calls on a collection. It takes the method name or array of names. The connection is optional if not set the default connection is used:

  // This is how grounddb uses this internally
  Ground.methodResume([
    '/' + self.name + '/insert',
    '/' + self.name + '/remove',
    '/' + self.name + '/update'
  ], self.connection);

The Ground.skipMethods is deprecated

Conflict handling IN the works - not ready for use yet

The conflict handling api is as follows:

Ground.now(); // Returns server timestamp works on both client and server

Additional api

Normally Ground Collections are cleaned up for local only data when subscriptions are ready. But sometimes we might want to local the data later eg. if the db is already populated.

  var groundList = new Ground.Collection('list', {
    cleanupLocalData: false
  });

  // Manually triggering a clean up of local only data
  // parameter query is optional, if passed, only local items that match query will be removed 
  groundList.removeLocalOnly(query); 

  // Clear local data - This will empty all local data
  groundList.clear();

Future

At the moment the conflict resolution is pretty basic last change recieved by server wins. This could be greatly improved by adding a proper conflict handler. For more details look at comment in server.js
Intelligent subscriptions - A way for the groundDB to keep the data most important for the user - and letting less important data go to match quota limit

Contributions

Feel free to send issues, pull requests all is wellcome

Kind regards Morten

EastonLee / db