Warning nextTick recursion error

[Ziggy Jonsson]

See pull-stream/stream-to-pull-stream#1 (comment) for example streamz production use. @bnoon I'd be very intested to hear how this is going and if you have been facing any immediate problems?

I have tested complex multiple-core (using node.cluster) data pipeline at around 20k rows per second and am very happy with the results so far. The only issue (hence this ticket) that I have experienced so far is nextTick warnings and maximum recursion errors when there is an aggregation of data somewhere in mid-pipeline (possibly coming from an SQL endpoint).
I
The working solution is to use setImmediate on the callbacks, allowing pending IO to finish (in the _transform prototype for streamz):

Stream.prototype._transform = function(d,e,cb) {
  var callback = function() {
    setImmediate(cb);
  }

I wonder whether this is a potentially more fundamental issue with the @joyent Node Readable, where setImmediate might be more appropriate than nextTick, i.e. https://github.com/joyent/node/blob/master/lib/_stream_readable.js#L532-L534

Anyway, this needs further investigation.