Abhrankan-Chakrabarti / llama_cpp_chatbot

Interactive Python chatbot powered by the `llama_cpp` library for text generation, offering customizable responses based on a user-provided GGUF model file.

Llama_CPP Chatbot

The Llama_CPP Chatbot is a Python chatbot that uses the llama_cpp library for text generation. It provides an interactive chatting experience where users can input text and receive responses generated by the chatbot. The chatbot's responses are based on a GGUF model file, which can be customized and fine-tuned for specific applications. The chatbot's parameters, such as top_k, top_p, temp, and repeat_penalty, can be adjusted to control the quality and diversity of the responses. The chatbot is designed to be easy to use and can be integrated into various applications that require natural language processing capabilities.

Installation

1. Clone the repository:

   git clone https://github.com/Abhrankan-Chakrabarti/llama_cpp_chatbot.git
   cd llama_cpp_chatbot

2. Install dependencies:

   pip install -r requirements.txt

Usage

1. Obtain a GGUF model file. Note that the GGUF file is not included in this repository and must be provided by the user.

2. Run the llama_cpp_chatbot.py script:

   python llama_cpp_chatbot.py

3. Follow the on-screen instructions to select the GGUF file.

4. Enter text to chat with the bot. Type "exit" to end the conversation.

Configuration

You can customize the chatbot's behavior by adjusting the parameters in the REPL loop (e.g., top_k, top_p, temp, repeat_penalty).

Contributing

Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.

License

This project is licensed under the MIT License - see the LICENSE file for details.