mlc-ai / web-llm

High-performance In-browser LLM Inference Engine

Home Page:https://webllm.mlc.ai

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

IndexedDB cache fails like the caches

DavidGOrtega opened this issue · comments

IndexedDB was added to avoid #257 However in the previously reported OS with Chrome browser the issue is similar. While the loading is completed at 100% the files are not there and load fails at the end.

Uncaught (in promise) Error: ArtifactIndexedDBCache failed to fetch: http://localhost:8082/dolphin-2_6-phi-2/MLC/params_shard_0.bin

The indexedDB cache only presents 4 out of 50 bin files

image

In chromium works as expected

image

Its related to the cache quota. For some reason in this OS and Chrome is 345MB

image

I can it force to be bigger and it works 🤔

image

Interestingly enough if I open a new tab without any URL the inspector shows a much larger limit

image

The problem is related on how Chrome manages quota and how much can be stored.
The maximum data allowed to store is severely restricted depending on opaque responses, incognito sessions and a very common setting that seems to be found enabled in MacOS by default:

If the user has enabled "Clear cookies and site data when you close all windows" in Chrome, the storage quota is significantly reduced to a maximum of approximately 300MB.

To avoid the restriction and be able to have bigger cache just change this setting on Chrome. To my mind this is a bump. Also que can confirm that IndexedDB is far better that caches as the limit is much higher.

image

I think this information would be very valuable in docs.

Really appreciate the findings! Will update the doc