VidBot is a chatbot that lets you ask questions about any video you want. Whether it's a movie, a lecture, or a tutorial, VidBot can help you find the information you need. You can use VidBot with videos stored on your computer or with YouTube links.
It uses the latest OpenAI GPT-4 for transcriptions and chat completions.
The app works as follows:
- Temp directory is created to store temporary files such as audio and transcriptions
- The user sends a video to the bot or points to a YouTube URL.
- The app extracts the audio from the video.
- The app sends the audio to OpenAI and waits for transcription.
- The app collates one or more transcriptions from OpenAI into a single transcription.
- The app sends the transcription to chatGPT 4 for correction.
- The app splits and inserts the correct transcription into a vectorDB for RAG.
- Finally, a RAG chatbot is created.
The goal of this app was to use two different OpenAI services, trancription and completions, for the creation of a video chatbot.
To install the app, you need to have Python 3 and pip installed on your system. Then, install the dependencies with:
pip3 install -r requirements.txtYou should create a .env file in the root directory of the app with the following variables:
OPENAI_API_KEY=<your open ai key>
OPENAI_API_TEMPERATURE=0.6
OPENAI_API_MODEL_NAME=gpt-4The app is built on top of Streamlit, but I provide a bootstrap file so you can run it simply with:
python3 bootstrap.pyExample URL: Introducing GPT-4
You can stop the application by pressing stop on the sidebar
- Langchain is poorly documented, lots of trial and error all around, specially to make custom prompts work
- If the transcription prompt is in question format or similar, GPT-4 will answer and therefore transcription will be overwritten by that answer. LOL
- AzureAI documentation and examples are fantastic
- Figuring a way to stop streamlit from within for mult-page app. I ended up using a sidebar page to stop the app.
See repository.