figure out why the default channel broke

[Alexander Cobleigh]

was it due to a malformatted message? a race condition? did someone accidentally fudge up the /default/metadata: {latest: <val>} value?

[Kira Oakley]

I dug into this. Notes:

• a lot of messages from one particular feed are missing; none of the usual peers seem to have them
• those missing messages are causing hyperdb to stall out on get and puts, because to do either hyperdb must traverse a subset of the nodes on a key prefix. It reaches a missing entry and gets stuck waiting forever for that node to be downloaded. Some nodes can become articulation points in the lookup graph, effectively severing reachability to lots of older messages as well.
• I talked to @mafintosh about it and he thinks he has some small code changes that would alleviate this

[Kira Oakley]

Handy trick for finding all missing nodes in a hyperdb:

var hyperdb = require('hyperdb')

var fn = '21b2b9ff201b01e6081709d82e6b81a5cf3a68d2cd5f092d0ffec58772642892'
var db = hyperdb(fn, { valueEncoding: 'json' })

db.ready(function () {
  for (var i=0; i < db._writers.length; i++) {
    var feed = db._writers[i]._feed
    for (var j=0; j < feed.length; j++) {
      var has = feed.has(j)
      if (!has) console.log('MISSING', i, j)
    }
  }
})

[Kira Oakley]

If we were using kappa-core, I think we could just have multifeed-index skip non-present nodes, and index them later. If we use unordered view modules like unordered-materialized-kv and the ilk, downloading them at a later time won't cause the views to become inconsistent.

[Alexander Cobleigh]

this has been resolved, but let's re-open this issue in case it reappears!