awslabs / rhubarb

A Python framework for multi-modal document understanding with Amazon Bedrock

Rhubarb

Rhubarb is a light-weight Python framework that makes it easy to build document understanding applications using Multi-modal Large Language Models (LLMs) and Embedding models. Rhubarb is created from the ground up to work with Amazon Bedrock and Anthropic Claude V3 Multi-modal Language Models, and Amazon Titan Multi-modal Embedding model.

What can I do with Rhubarb?

Visit Rhubarb documentation.

Rhubarb can do multiple document processing tasks such as

• ✅ Document Q&A
• ✅ Streaming chat with documents (Q&A)
• ✅ Document Summarization
  • 🚀 Page level summaries
  • 🚀 Full summaries
  • 🚀 Summaries of specific pages
  • 🚀 Streaming Summaries
• ✅ Structured data extraction
• ✅ Named entity recognition (NER)
  • 🚀 With 50 built-in common entities
• ✅ PII recognition with built-in entities
• ✅ Figure and image understanding from documents
  • 🚀 Explain charts, graphs, and figures
  • 🚀 Perform table reasoning (as figures)
• ✅ Document Classification with vector sampling using multi-modal embedding models
• ✅ Logs token usage to help keep track of costs

Rhubarb comes with built-in system prompts that makes it easy to use it for a number of different document understanding use-cases. You can customize Rhubarb by passing in your own system prompts. It supports exact JSON schema based output generation which makes it easy to integrate into downstream applications.

• Supports PDF, TIFF, PNG, JPG files (support for Word, Excel, PowerPoint, CSV, Webp, eml files coming soon)
• Performs document to image conversion internally to work with the multi-modal models
• Works on local files or files stored in S3
• Supports specifying page numbers for multi-page documents
• Supports chat-history based chat for documents
• Supports streaming and non-streaming mode

Installation

Start by installing Rhubarb using pip.

pip install pyrhubarb

Usage

Create a boto3 session.

import boto3
session = boto3.Session()

Call Rhubarb

Local file

from rhubarb import DocAnalysis

da = DocAnalysis(file_path="./path/to/doc/doc.pdf", 
                 boto3_session=session)
resp = da.run(message="What is the employee's name?")
resp

With file in Amazon S3

from rhubarb import DocAnalysis

da = DocAnalysis(file_path="s3://path/to/doc/doc.pdf", 
                 boto3_session=session)
resp = da.run(message="What is the employee's name?")
resp

For more usage examples see cookbooks.

Security

See CONTRIBUTING for more information.

License

This project is licensed under the Apache-2.0 License.