khozyainov / registry-tabby

🧑‍🔬 Tabby Registry

Completion models (--model)

We recommend using

• For 1B to 3B models, it's advisable to have at least NVIDIA T4, 10 Series, or 20 Series GPUs.
• For 7B to 13B models, we recommend using NVIDIA V100, A100, 30 Series, or 40 Series GPUs.

Model ID	License
khozyainov/StarCoder-1B	BigCode-OpenRAIL-M
khozyainov/StarCoder-3B	BigCode-OpenRAIL-M
khozyainov/StarCoder-7B	BigCode-OpenRAIL-M
khozyainov/CodeLlama-7B	Llama 2
khozyainov/CodeLlama-13B	Llama 2
khozyainov/DeepseekCoder-1.3B	Deepseek License
khozyainov/DeepseekCoder-6.7B	Deepseek License
khozyainov/DeepseekCoder-6.7B-instruct_Q4_K_M	Deepseek License
khozyainov/DeepseekCoder-6.7B-instruct_Q5_K_M	Deepseek License
khozyainov/DeepseekCoder-6.7B-instruct_Q5_0	Deepseek License
khozyainov/DeepseekCoder-6.7B-instruct_Q8_0	Deepseek License

Chat models (--chat-model)

To ensure optimal response quality, and given that latency requirements are not stringent in this scenario, we recommend using a model with at least 3B parameters.

Model ID	License
khozyainov/WizardCoder-3B	BigCode-OpenRAIL-M
khozyainov/Mistral-7B	Apache 2.0
khozyainov/DeepseekCoder-6.7B-instruct_Q4_K_M	Deepseek License
khozyainov/DeepseekCoder-6.7B-instruct_Q5_K_M	Deepseek License
khozyainov/DeepseekCoder-6.7B-instruct_Q5_0	Deepseek License
khozyainov/DeepseekCoder-6.7B-instruct_Q8_0	Deepseek License