createReadStream seems to crash the program/make the file handles hang permanently
robobun opened this issue · comments
createReadStream (on its own or when given a specific file handle) seems to take permanent ownership of its file handle and not let it be closed naturally/manually
bun version: 1.0.20
operating system: debian on wsl 2 (5.10.102.1-microsoft-standard-WSL2)
ulimit size: 1024
code:
import { createReadStream, close } from "fs";
import { open, writeFile } from "fs/promises";
await writeFile("mlem", Buffer.alloc(2 ** 18));
const openAndReadFile = () => new Promise(async (resolve, reject) => {
const fd = await open("mlem", 'r');
if (typeof fd === "number")
createReadStream("", { fd, autoClose: false, emitClose: false })
.on("data", chunk => { })
.on("end", () => {
close(fd, error => void (error ? reject(error) : resolve()));
})
.on("error", reject);
else
fd.createReadStream({ autoClose: false, emitClose: false })
.on("data", chunk => { })
.on("end", () => {
fd.close().then(resolve).catch(reject);
})
.on("error", reject);
});
for (let i = 0; i < 3000; ++i) {
console.log("opened", i);
await openAndReadFile();
console.log("closed");
}
output when ran from node.js:
opened 0
closed
...
opened 2999
closed
output when ran from bun:
opened 0
closed
...
opened 2041
closed
1 | export default "native";
^
EMFILE: Too many open files
errno: -24
syscall: "dup"
fd: 13
at native:1:1
at #internalConstruct (node:1:62)
at _read (node:1:50)
at #internalRead (node:1:465)
at node:1:1993
at maybeReadMore (native:1:1)
at processTicksAndRejections (:61:39)
Originally reported on Discord: createReadStream seems to crash the program/make the file handles hang permanently
I am facing the same issue with my express server.
Here is a test repository:
https://github.com/lmachens/bun-file-not-closed
Each time a file is opened on the server (e.g. to serve the index.html), the file handlers count increases.).
I ran your test (bun 1.0.21) and did not see the error. I am running on a Mac, so not sure if it is the different platform or the different bun version.
I ran your test (bun 1.0.21) and did not see the error. I am running on a Mac, so not sure if it is the different platform or the different bun version.
I have the same issues with 1.0.21, but on Ubuntu and with the official docker image.
I tried out 1.0.22 (bun upgrade --canary && bun install
) but it's still an issue
This still needs to be fixed, but in the meanwhile, you can use Bun.file().stream()
instead:
const foo = await Bun.file("foo.txt").stream();
This still needs to be fixed, but in the meanwhile, you can use
Bun.file().stream()
instead:const foo = await Bun.file("foo.txt").stream();
Thx. Any advice how to apply this for Express?
This still needs to be fixed, but in the meanwhile, you can use
Bun.file().stream()
instead:const foo = await Bun.file("foo.txt").stream();Thx. Any advice how to apply this for Express?
I'm not sure about Express. There's more info about this in the docs though, if you want to look there.