English / ็ฎไฝไธญๆ / ๆฅๆฌ่ช
๐ฃ๏ธ Chat with LLM like Vicuna totally in your browser with WebGPU, safely, privately, and with no server. Powered By web-llm.
-
๐ค Everything runs inside the browser with no server support and is accelerated with WebGPU.
-
โ๏ธ Model runs in a web worker, ensuring that it doesn't block the user interface and providing a seamless experience.
-
๐ Easy to deploy for free with one-click on Vercel in under 1 minute, then you get your own ChatLLM Web.
-
๐พ Model caching is supported, so you only need to download the model once.
-
๐ฌ Multi-conversation chat, with all data stored locally in the browser for privacy.
-
๐ Markdown and streaming response support: math, code highlighting, etc.
-
๐จ responsive and well-designed UI, including dark mode.
-
๐ป PWA supported, download it and run totally offline.
-
๐ To use this app, you need a browser that supports WebGPU, such as Chrome 113 or Chrome Canary. Chrome versions โค 112 are not supported.
-
๐ป You will need a GPU with about 6.4GB of memory. If your GPU has less memory, the app will still run, but the response time will be slower.
-
๐ฅ The first time you use the app, you will need to download the model. For the Vicuna-7b model that we are currently using, the download size is about 4GB. After the initial download, the model will be loaded from the browser cache for faster usage.
-
โน๏ธ For more details, please visit mlc.ai/web-llm
-
[โ ] LLM: using web worker to create an LLM instance and generate answers.
-
[โ ] Conversations: Multi-conversation support is available.
-
[โ ] PWA
-
[] Settings:
- ui: dark/light theme
- device:
- gpu device choose
- cache usage and manage
- model:
- support multi models: vicuna-7bโ RedPajama-INCITE-Chat-3B []
- params config: temperature, max-length, etc.
- export & import model
- Click
, follow the instructions, and finish in just 1 minute.
- Enjoy it ๐
git clone https://github.com/Ryan-yang125/ChatLLM-Web.git
cd ChatLLM-Web
npm i
npm run dev
