Rivridis / LLM-Assistant

Locally running LLM with internet access

LLM-Assistant

LLM-Assistant is a browser interface based on Gradio that interfaces with local LLMs to call functions and act as a general assistant.

Features

• Works with any instruct-finetuned LLM
• Can search for information (RAG)
• Knows when to call functions
• Realtime mode for working across the system
• Answers question from PDF files

Roadmap

• Voice access
• More functions

Current Bugs

• Rare crashes

Changelog

• Fixed search feature
• Youtube video search
• File Upload

Setup

Setup on Windows 10/11

1. Clone repo to a virtual environment
2. Install requirements.txt
3. Download and place LLM model in model folder
4. Run main.py

Usage

• Use Assistant mode for general chat, and calling functions to execute like playing music, as well as PDF question answering
• Use Realtime mode for editing a word document or replying to an email in realtime, directly by copying a selection and waiting for the output. The output gets auto pasted at cursor location.

Images