Problems with large disk subsystems in Firefox
pkimpel opened this issue · comments
Reported by Paul Cumberworth on 2015-06-05:
I still haven’t got my head around large Disk subsystems – I used to get a dialog box in Firefox asking me for permission to exceed 50Mb I think it was, but that doesn’t seem to happen anymore. For now I’m just trying to keep my file systems small.
I have also noticed this problem, especially when multiple disk subsystems have been created. Firefox docs indicate there is no size limit for IndexedDB databases, but this appears to be a limit imposed on off-line applications (i.e., those in the App Cache).
It appears that Firefox (as of version 38, at least) has a fixed upper limit on 2GB on off-line data for each web origin (i.e., web site). This does not appear to be documented anywhere, but postings on StackOverflow.com reveal that others have hit this limit.
The real problem here is that the method we are using to store disk sector data in IndexedDB is very space-inefficient. The SQLite database for a disk subsystem that has somewhere in the range of 40-60MB of actual data is occupying 1.8GB of disk space. We need to experiment with alternatives that will give more a space-efficient behaivor.
Fixed in release 1.03. On a write, B5500DiskUnit.js was extracting individual disk sectors from the IO Unit's buffer using Uint8Array.subarray(), which was apparently returning an object that referenced the underlying arrays. When IndexedDB did a structured clone of that object (which it is required to do by the specs), the 16KB IO Unit buffer got included in the clone and written to the underlying database, albeit in compressed form. This was corrected by copying the sector data to a local 240-byte Uint8Array and doing the IDB put() on the local copy. See the blog and forum for details.