nihalalizada / conversational-agent-langchain

FastAPI Backend for a Conversational Agent using Aleph Alpha, (Azure) OpenAI, Langchain and a VectorDB

Conversational Agent

This is a Rest-Backend for a Conversational Agent, that allows to embedd Documentes, search for them using Semantic Search, to QA based on Documents and do document processing with Large Language Models.

• Conversational Agent
  • Recent Updates
  • Quickstart
  • Redoc API Documentation
  • Project Description
  • Semantic Search
  • Architecture
  • Components
  • Available LLM Backends
  • Deployment
  • Secret Management
  • Installation & Development Backend
  • Development Frontend
  • Vector Database
  • Qdrant API Key

Recent Updates

• Added Qdrant as a vector database.
• Added GPT4All as a LLM Backend. It does not work if you build a docker image on a mac device.

Quickstart

To run the complete system with docker use this command:

git clone https://github.com/mfmezger/conversational-agent-langchain.git
cd conversational-agent-langchain

Create a .env file from the .env-template and set the qdrant api key. For tests just set it to test. QDRANT_API_KEY="test"

Then start the system with

  docker compose up -d

Then go to http://localhost:8001/docs or http://localhost:8001/redoc to see the API documentation.

Redoc API Documentation

Project Description

This project is a conversational agent that uses Aleph Alpha and OpenAI Large Language Models to generate responses to user queries. The agent also includes a vector database and a REST API built with FastAPI.

Features

• Uses Aleph Alpha and OpenAI Large Language Models to generate responses to user queries.
• Includes a vector database to store and retrieve information.
• Provides a REST API built with FastAPI for easy integration with other applications.
• Has a basic gui.

Semantic Search

Semantic search is an advanced search technique that aims to understand the meaning and context of a user's query, rather than matching keywords. It involves natural language processing (NLP) and machine learning algorithms to analyze and interpret user intent, synonyms, relationships between words, and the structure of content. By considering these factors, semantic search improves the accuracy and relevance of search results, providing a more intuitive and personalized user experience.

Architecture

Components

Langchain is a library for natural language processing and machine learning. FastAPI is a modern, fast (high-performance) web framework for building APIs with Python 3.7+ based on standard Python type hints. A Vectordatabase is a database that stores vectors, which can be used for similarity searches and other machine learning tasks.

Available LLM Backends

• Aleph Alpha Luminous
• GPT4All
• (Azure) OpenAI

Deployment

If you want to use a default token for the LLM Provider you need to create a .env file. Do this by copying the .env-template file and add the necessary api keys.

If you are working in an envoironment with internet connection the easiest way is to use this command:

docker compose -f docker-compose-hub.yml up

This will pull the image from docker hub and run it. Instead of building it on your local machine.

If you want to build the image on your local machine you can use this command:

docker compose up

Secret Management

Two ways to manage your api keys are available, the easiest approach is to sent the api token in the request as the token. Another possiblity is to create a .env file and add the api token there. If you use OpenAI from Azure or OpenAI directly you need to set the correct parameters in the .env file.

Installation & Development Backend

First install Python Dependencies:

pip install poetry
poetry install

Start the complete system with:

docker compose up -d

To run the Qdrant Database local just run:

docker compose up qdrant

To run the Backend use this command in the root directory:

poetry run uvicorn agent.api:app --reload

To run the tests you can use this command:

poetry run coverage run -m pytest -o log_cli=true -vvv tests

Development Frontend

To run the Frontend use this command in the root directory:

poetry run streamlit run gui.py

Vector Database

Qdrant Dashboard is available at http://localhost:6333/dashboard. There you need to enter the api key.

Qdrant API Key

To use the Qdrant API you need to set the correct parameters in the .env file. QDRANT_API_KEY is the API key for the Qdrant API. And you need to change it in the qdrant.yaml file in the config folder.