Broken streams again
mhartington opened this issue · comments
Seems the same issue from #8 is back again.
Between 2.8.2 and 2.8.3, we're seeing streams would stop being processed by tar, with minpass being the cause. Similar comment was made in #8
Our issue was that tar 4.4.10 would apparently stop processing a stream, leaving the Node.js event loop empty and allowing the process to exit.
Same here, minipass@>=2.8.2 is broken.
Downgrading minipass also doesn't fix the issue for me.
It's definitely not the same problem as #8. I've tracked down the root cause, will revert the change that's breaking tar.
Need to figure out another way to handle tee-piping in the presence of already buffered data, I guess.
Pushed a 2.8.5 that makes minipass play nice with node-tar again.
The unfortunate part is that it means it doesn't play nice with minipass-fetch, which is what prompted this change in the first place. But way fewer people use that, so it's lower priority.
@isaacs Thank you for both addressing this problem so quickly and your continued maintenance of this module.
No problem, sorry for the hiccup :)
I updated the docs to explain how to do tee streams safely without losing data. I don't think there's a way to make this work properly to get data to both places:
src.write('foo')
src.pause()
src.pipe(d1)
src.pipe(d2)
src.resume()
So you just have to do this:
src.write('foo')
const t = new Minipass()
t.pipe(d1)
t.pipe(d2)
src.pipe(t)
In any event, the overhead of an additional object is pretty minimal, compared with the overhead of the bookkeeping and heuristics to determine whether we should resume on pipe() or not.