OPFS without Workers
runspired opened this issue · comments
OPFS only requires secure contexts (https) not workers to run. I'm curious if the constraint around this listed in the README is just outdated or if this is due to requiring use of FileSystemSyncAccessHandle
Sync access handles are required in this VFS and they are only available in workers. To the very best of my knowledge, OPFS itself is only available in workers - do you have a link to documentation which says otherwise? (Edit: corrected in follow-up comments)
Edit:
https://developer.mozilla.org/en-US/docs/Web/API/File_System_API/Origin_private_file_system
Says:
This feature is available in Web Workers.
@sgbeal I've been using OPFS from main thread quite extensively in other work. The only feature not available on the main thread is the sync access handle, the async APIs for OPFS are accessible from any secure context.
Since the sync access handle is the only API which allows incremental loading of the file starting from a specific byte offset, I'm assuming sqlite wasm uses that extensively for paging etc, but other than that feature the async APIs available everywhere enable all the same things.
I'm assuming sqlite wasm uses that extensively for paging etc,
sqlite has a 100% synchronous I/O API, so requires the sync access handles. The wa-sqlite project has several OPFS VFS variants which take different approaches, at least one of which can be used in the main thread:
Ours, however, requires the sync access handles.
but other than that feature the async APIs available everywhere enable all the same things.
And i mis-read the MDN note on that topic:
This feature is available in Web Workers.
does not say "ONLY in Web Workers."
When setting up the OPFS VFS, the library simply probes for the available APIs - it doesn't actually care if it's running in a worker or not, so long as the required APIs can be found. Unfortunately, browser developers feel that it's their job to enforce that page developers never block the main thread, so browsers very specifically do not permit sync access handles in the main thread :(.
the async APIs available everywhere enable all the same things
@runspired OPFS writing without using synchronous access handles works reasonably well for writing complete files, but it is extremely inefficient at modifying files (large files in particular) which is what database apps commonly do. So it's arguable that it doesn't "enable all the same things".
Also, I believe Safari and Android Chrome still don't support FileSystemWritableFileStream so that wouldn't meet "available everywhere".
For the purposes of database persistence, OPFS without sync access handles isn't a great persistence solution in general. There might be some narrow use cases where it makes sense, e.g. read-only.
@rhashimoto oh totally, I agree with all of that assessment. I was more curious if the sync handle was the constraint here or not (it is). The downside of the way that sync handle is only available in dedicated workers is that sqlite is most useful if used inside a SharedWorker. I'm curious if we can push the technical committee to get the dedicated worker constraint lifted.