TotallyInformation / node-red-contrib-gun-db

Collection of nodes to use Gun.js with Node-RED *Now in Beta*

node-red-contrib-gun-db

Unfortunately, GunDB did not prove as robust and useful as anticipated for Node-RED and so this repo will not be developed further and so is being archived. Please feel free to fork if you wish.

---

Collection of nodes to use Gun.js with Node-RED

Based on a suggestion from thinkbig1979 in the Node-RED Forum.

Prerequisites

To use this Node, you will need to:

• use at least Node-RED v1
• use a recent browser (not IE)

Installation

Install production versions from Node-RED's palette manager.

Install development versions from GitHub:

cd ~/.node-red
npm install TotallyInformation/node-red-contrib-gub-db

What Works

• Use the Gun Listen node to listen for any changes to or beneath the given path. Uses Gun's .on() or .open() functions.
• Use the Gun Out node to trigger a one-off output based on fixed settings or using the msg.topic to set the path. Uses Gun's .once() or .load() functions.
• Use the Gun Update node to add or change data. Uses Gun's .put() or .set() functions.
• Use the Gun Cofig configuration node to specify a new instance of Gun.js that is used by the other nodes. Each instance creates a new physical database locally and, optionally, is able to link to a remote instance for peer-to-peer synchronisation.

To Do

• See the To Do sections for the nodes as well.

• Add a Gun server

  So that Node-RED can be used as a Gun backend as well. In this case, you would add GunJS to your front-end (browser) code and connect it to the Node-RED server so that data is automatically synchronised.

  Will also need to try to implement the .bye() handling.

• Find a better way at handling JavaScript Array's & functions to make the output better able to handle random JavaScript data.

• Add more of Gun.js's features:

  • Add an unset feature to allow removal of unordered data added via set.
  • Gun.js user handling
  • gun.later(cb, seconds)
  • Utils: Gun.node.is(data), Gun.node.soul(data), Gun.node.ify(json)

• Still more (note these will probably never be implemented unless someone really wants to ask for them or is able to help with coding and testing)

  • SEA, RAD, JOY API's
  • Use other data adapters rather than just the filing system
  • Proper delete function (don't think this is possible with current Gun.js API's)

Use Cases

2 distinct use-cases here at least:

1. A server that allows front-end's (whether data driven from Node-RED or otherwise) to sync/share data.
2. A DB for Node-RED itself to use. Which doesn't, of course, need the server component.

Both have significant potential benefits. Especially given the ability to subscribe to changes.

Potentially also useful for a Node-RED datastore. (Maybe. The two weaknesses listed below may yet preclude this. TBD).

Strengths

• Lightweight
• NoSQL, document style, eventually consistent
• Use stand-alone or in peer-to-peer sync, no central server required
• Use in browser and/or node.js
• Subscribe to data changes.
• Partial document updates
• Capable of graph data structures.
• Can handle JavaScript circular data references

Weaknesses

• No proper delete function - difficult to completely clean out data

  To properly delete an entire hierarchy, you would have to manually walk the data and nullify each element.

  To actually delete data, you would have to delete from the filing system.

• Cannot handle all native JavaScript object types (e.g. array's)

  You cannot simply throw a random JavaScript object at Gun.js's put or set function.

• Inconsistent and unexpected behaviours

  The Gun.js documentation has a significant number of comments about unexpected behaviours.

  Here are a couple of the worst that I found during development:

  • No delete

    Covered this already but the worst thing is that if you have a deep hierarchy of nodes and then set one of the nodes to null as suggested in the documentation, you can still access data lower in the hierarchy since that hasn't been removed. Whilst this is obviously very fast, it makes Gun.js totally unsuitable for some uses.

  • Creating a deep entry does not create intermediate nodes

    Using something like Gun.get('DB/some/deep/path').put({a:1,b:2}) does NOT create the intermediate nodes!

    So you cannot then do Gun.get('DB/some').once((data, key) => {console.log(key,data)}) because this will return undefined for the data.

    To get a more anticipated outcome, you have to do Gun.get('DB').get('some').get('deep').get('path').put({a:1,b:2}).

  • SET allows you to add a string/number/true/false/null to the data but PUT does not

  • If you use SET to add a string/number/true/false/null to the data, you cannot directly query the added data

    If the added data was at DB/some/deep/path/kb5b6jxu3lnlClU for example, you cannot do Gun.get('DB/some/deep/path/kb5b6jxu3lnlClU').once((data, key) => {console.log(key,data)}) as this will return undefined as the data. Instead you have to query the parent and then reference the property Gun.get('DB/some/deep/path').once((data, key) => {console.log(key,data.kb5b6jxu3lnlClU)})

    However, if you use SET to add an Object, you CAN query it directly!

  • UNSET is inconsistent with everything else!

    I still have not found a way to use it without keeping a reference to the way you created it as in the only single example code I've found. This is OK if the entry is an object because you can GET it. But if it isn't, there appears to be no way to get a reference.

    Without it, there doesn't appear to be any way to remove something added via SET.

Installation

Install from npm using Node-RED's Palette Manager or from the command line using npm.

Nodes

Gun: Out - get-once

Uses the .once Gun function to output data from a given path.

You can choose to output a msg for each property of the given db/path or a single message.

Note that, by default, Gun only directly returns a single level of the hierarchy and references to the next level. Should you want the whole hierarchy, you can choose the "Output the whole hierarchy" option.

Additionally, you can choose whether to include Gun's metadata (the _ property) or not. If outputting the whole hierarchy, the metadata is not available.

Gun: Listen - get

Listen for new or changed data on or below the given path. Uses the Gun .on function.

Gun: Update - put, set

Put data to the specified path. Optionally use .set instead of .put.

To do

• Output a warning if anything but an object is put
• Output a warning if an Array being set
• Consider adding a flag that does a once after a put/set and returns the data in the output msg
• Consider an option that, if the input is an array, offers to loop over the array doing a put or set for each entry.

Gun Config

Configuration node primarily to give common access to the Gun() instance. Optionally, create a Gun.js server instance.

To Do

• Ensure that every config instance uses a different physical storage location.

Other Info

The Gun data is set to <userDir>/_gun/.

All the main nodes (not the config node) require a "Db" to be set. This is a top-level Gun element. This forces all user data to be stored below the top-level of the hierarchy which Gun treats slightly differently.

References

• Gun.js
• Gun.js GitHub
• Gun.js questions on StackOverflow
• Couple of useful articles about Gun.js on Medium. 1, 2

---

Please also check out my blog, Much Ado About IT, it has information about all sorts of topics, mainly IT related, including Node-RED.