isaacs / minipass

A stream implementation that does more by doing less

Home Page:https://twitter.com/izs/status/1174465160737509376

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Broken streams again

mhartington opened this issue · comments

Seems the same issue from #8 is back again.

Between 2.8.2 and 2.8.3, we're seeing streams would stop being processed by tar, with minpass being the cause. Similar comment was made in #8

Our issue was that tar 4.4.10 would apparently stop processing a stream, leaving the Node.js event loop empty and allowing the process to exit.

Same here, minipass@>=2.8.2 is broken.

commented

Downgrading minipass also doesn't fix the issue for me.

It's definitely not the same problem as #8. I've tracked down the root cause, will revert the change that's breaking tar.

Need to figure out another way to handle tee-piping in the presence of already buffered data, I guess.

Pushed a 2.8.5 that makes minipass play nice with node-tar again.

The unfortunate part is that it means it doesn't play nice with minipass-fetch, which is what prompted this change in the first place. But way fewer people use that, so it's lower priority.

commented

Issue is fixed with minipass@2.8.5.
Happy to see this again, thanks isaac!
fixed

@isaacs Thank you for both addressing this problem so quickly and your continued maintenance of this module.

No problem, sorry for the hiccup :)

I updated the docs to explain how to do tee streams safely without losing data. I don't think there's a way to make this work properly to get data to both places:

src.write('foo')
src.pause()
src.pipe(d1)
src.pipe(d2)
src.resume()

So you just have to do this:

src.write('foo')
const t = new Minipass()
t.pipe(d1)
t.pipe(d2)
src.pipe(t)

In any event, the overhead of an additional object is pretty minimal, compared with the overhead of the bookkeeping and heuristics to determine whether we should resume on pipe() or not.