LotfiMEZIANI / rag-chatbot

In this project, we will implement a chatbot ([RAG](https://www.promptingguide.ai/techniques/rag) model) that can answer questions about Next.js documentation.

NextJs Documentation Assistant (RAG Chatbot)

Overview

In this project, we will implement a chatbot (RAG model) that can answer questions about Next.js documentation.

1. Documentation Acquisition: Downloading HTML content from the Next.js Official Documentation.
2. HTML Scrapping: Extracting critical data, focusing on the article tag from each page.
3. Data Processing: Tokenizing and vectorizing the collected information.
4. Data Indexing: Storing the processed data in a Pinecone index for efficient retrieval.
5. Chatbot Creation: Using LangChain in conjunction with OpenAI models and the Pinecone index to develop a responsive chatbot.

Getting Started

Get started

Prerequisites

• Python 3.10.13
• OpenAI API key
• Pinecone API key

Setup the project

1. Create a python virtual environment:

python -m venv .venv

2. Activate the virtual environment:

source .venv/bin/activate

3. Install dependencies:

pip install -r requirements.txt

4. Duplicate .env.template to create .env..

   Remember to populate it with your OpenAI API key, Pinecone API key, and Pinecone environment name.

5. To download the sources, run the following command in your terminal:

python download_sources.py

6. To vectorize the sources, run the following command in your terminal:

python vectorize_sources.py

7. To start the assistant, run the following command in your terminal:

streamlit run main.py