Propagate FileReader Errors

[Garrett Johnson]

I this issue when trying to process a large (1.0+GB) rosbag file in the browser -- when the FileReader hits an error the bag handle never responds because onload is never called.

The error can be propagated by adding something like the following code in this function:

reader.onerror = function() {
    reader.onload = null;
    reader.onerror = null;

    setImmediate(cb, reader.error, null);
}

I'd submit a PR but I'm not certain how to reproduce the issue and it seems to happen intermittently. I'm only seeing the issue in Chrome and seeing that the error returned by the reader is DOMException: The requested file could not be read, typically due to permission problems that have occurred after a reference to a file was acquired.

I'm loading and opening the bag like so:

import { open } from 'rosbag';

fetch(url, { credentials: 'same-origin' })
    .then(res => res.blob())
    .then(blob => open(new File([blob], 'temp.bag')))
    .then(() => { /* ... nothing happens ... */ })

Ultimately it seems to be a browser problem but it would be nice to propagate the error. Admittedly my experience with the FileReader API is limited -- maybe you guys have a bit more insight?

Thanks!

[JP Posma]

Interesting! I don't think we've loaded such big bag files in memory — usually we'd read them off the filesystem directly when they're that big. Any chance you might be hitting some memory limits? Maybe the bit about permissions is a red herring (especially since they say "typically due to")?

[Garrett Johnson]

Yeah I'd expect that downloading a 1GB file blob would fall into the "not recommended" category -- we've been using smaller bags for our work until now.

... usually we'd read them off the filesystem directly when they're that big.

We're considering building out an Electron app to support cases where our bag data is so large. Is that how you're reading files directly off the file system? Or are there other ways you're using to read files directly in the browser (such as the file input field)?

Maybe the bit about permissions is a red herring (especially since they say "typically due to")?

The error message definitely struck me as a generic one so I'd believe this.

Thanks!

[JP Posma]

We're mostly using the drag and drop api. It gives you a File object with the only difference being that the data remains on disk. You can pass it into open from rosbag just like above.

[JP Posma]

And yeah a file input field works the same way.

[Jacob Bandes-Storch]

Regardless of the cause, you're right that it seems to make a lot of sense to propagate the onerror event from the reader. As for reproducing it for testing, it does seem surprisingly tricky to force a failure, but it would work to instead just mock out the readAsArrayBuffer function for testing so that it just asynchronously calls the onerror.

[Katerhong]

so, What is the conclusion?
I feel confused

[Jacob Bandes-Storch]

The original issue was fixed and published in version 1.3.2. If you are having trouble, I would recommend filing a new issue. By the way, you might also want to check out @foxglove/rosbag, a fork of this project with some updates to the API and more bug fixes.