Forked from https://github.com/imartinez/privateGPT (with some small fixes)
Ask questions to your documents without an internet connection, using the power of LLMs. 100% private, no data leaves your execution environment at any point. You can ingest documents and ask questions without an internet connection!
To use this software, you must have Python 3.10 or later installed. Earlier versions of Python will not compile.
pip install -r requirements.txt
Download https://huggingface.co/orel12/ggml-gpt4all-j-v1.3-groovy/resolve/main/ggml-gpt4all-j-v1.3-groovy.bin and put it in a folder models
.
Put any and all your files into the source_documents
directory
The supported extensions are:
.csv
: CSV,.docx
: Word Document,.doc
: Word Document,.enex
: EverNote,.eml
: Email,.epub
: EPub,.html
: HTML File,.md
: Markdown,.msg
: Outlook Message,.odt
: Open Document Text,.pdf
: Portable Document Format (PDF),.pptx
: PowerPoint Document,.ppt
: PowerPoint Document,.txt
: Text file (UTF-8),
Run the following command to ingest all the data.
python ingest.py
It will create a db
folder containing the local vectorstore. Will take 20-30 seconds per document, depending on the size of the document.
You can ingest as many documents as you want, and all will be accumulated in the local embeddings database.
If you want to start from an empty database, delete the db
folder.
Note: during the ingest process no data leaves your local environment. You could ingest without an internet connection, except for the first time you run the ingest script, when the embeddings model is downloaded.
In order to ask a question, run a command like:
python privateGPT.py
And wait for the script to require your input.
> Enter a query:
Hit enter. You'll need to wait 20-30 seconds (depending on your machine) while the LLM model consumes the prompt and prepares the answer. Once done, it will print the answer and the 4 sources it used as context from your documents; you can then ask another question without re-running the script, just wait for the prompt again.
Note: you could turn off your internet connection, and the script inference would still work. No data gets out of your local environment.
Type exit
to finish the script.
The script also supports optional command-line arguments to modify its behavior. You can see a full list of these arguments by running the command python privateGPT.py --help
in your terminal.
Selecting the right local models and the power of LangChain
you can run the entire pipeline locally, without any data leaving your environment, and with reasonable performance.
ingest.py
usesLangChain
tools to parse the document and create embeddings locally usingHuggingFaceEmbeddings
(SentenceTransformers
). It then stores the result in a local vector database usingChroma
vector store.privateGPT.py
uses a local LLM based onGPT4All-J
orLlamaCpp
to understand questions and create answers. The context for the answers is extracted from the local vector store using a similarity search to locate the right piece of context from the docs.GPT4All-J
wrapper was introduced in LangChain 0.0.162.