local-inference

There are 2 repositories under local-inference topic.

SJTU-IPADS / PowerInfer
High-speed Large Language Model Serving on PCs with Consumer-grade GPUs
bamboo-7b falcon large-language-models llama llm llm-inference local-inference
Language:C++ 7054
efeslab / fiddler
Fast Inference of MoE Models with CPU-GPU Orchestration
llm llm-inference mixtral-8x7b mixture-of-experts local-inference
Language:Python 147
BorjaOteroFerreira / IALab-Suite
Tool for test diferents large language models without code.
flask-api llm llm-inference chat-application inference-api large-language-models llama-cpp-python llama2 api-rest llama2-7b llamacpp mixtral-8x7b local-inference
Language:Python 15
yas-sim / openvino-llm-chatbot-rag
LLM chatbot example using OpenVINO with RAG (Retrieval Augmented Generation).
chatbot llm openvino rag retrieval-augmented-generation dolly2 edge-computing edge-inference llama2 intel langchain natural-language-processing offline cloud-free local-inference huggingface neural-chat
Language:Python 3