Giters

mlc-ai / web-llm

High-performance In-browser LLM Inference Engine

Home Page:https://webllm.mlc.ai

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Model request: new Mistral 7B with 32K context

flatsiedatsie opened this issue · comments

commented

Recently a new version of Mistral was announced which has a much larger 32K context:
https://www.reddit.com/r/LocalLLaMA/comments/1blzrfp/new_mistral_model_announced_7b_with_32k_context/

This model could greatly increase the ability to create good, coherent summaries of documents. I hope it could become part of the "Web-LLM suite".

commented

I realized that perhaps my request sounds odd, since LMM is probably made to allow you to add any new model yourself? Could I theoretically get Mistral weith 32K context running myself?

The issue for me is that I'm a n00b and don't have a lot of knowledge in this area. I did try, but so far I've resorted to integrating Web-LLM into a project by copying the online chat example and hacking the hell out of that. Which means I'm limited in which models I can use, since I can only use the ones available in the demo.

So my question to add the model is effectively: could the model becomes available in the online demo? :-)