DokGPT implements advanced LLM prompting for organizing and discussing PDFs. The repository uses tools such as PyPDF2 and pdf2image for PDF processing, Google Vision for text extraction from images, nltk for text fragment/chunk extraction, Weaviate for dense vector search and embedding handling, and FeatureBase for back of the book indexing and graph traversal of questions.
The project is installed, configured and run locally from the command line. You will need a Google Cloud account with Vision enabled, an OpenAI account, a FeatureBase cloud account and a Weaviate cloud account to run the code.
The process of indexing a document is divided into three main steps:
- A PDF (and later webpages) are processed into images. These images are then used for text extraction with the help of Google Vision. Early concepts of this idea have been previously implemented at Mitta.us.
- The extracted text is chunked by nltk and stored in Weaviate, which handles the ada-002 embeddings using OpenAI's APIs. During this indexing step, a back-of-the-book index is also created, and the keyterms are stored in an inverted index in FeatureBase along with the text chunk's UUIDs from Weaviate and the page numbers.
- During interactions with the LLM, Weaviate and FeatureBase are used together to determine the most relevant text to use for building prompts. Texts are passed around as documents and augmented with queries to the LLM as well as use of any documents that are related through the inverted index in FeatureBase.
To use Weaviate and FeatureBase with DocGPT, you will need to set up accounts with their respective cloud providers. Follow the instructions below to get started:
Go to the Weaviate Cloud website and sign up for an account.
Once you have signed up, you will be taken to the dashboard. Click on the "Create Weaviate Cluster" button to create a new cluster.
Follow the on-screen instructions to configure your cluster. You will need to choose a provider, region, and cluster size. You can also choose to enable backups and enable SSL.
Once your cluster is set up, you can access it from the dashboard.
Go to the FeatureBase Cloud website and sign up for an account.
Once you have signed up, you will be taken to the dashboard. Click on the "Create FeatureBase Cluster" button to create a new cluster.
Follow the on-screen instructions to configure your cluster. You will need to choose a provider, region, and cluster size. You can also choose to enable backups and enable SSL.
Once your cluster is set up, you can access it from the dashboard.