jamesmcroft / compress-gpt

Self-extracting GPT prompts for ~70% token savings

CompressGPT

Self-extracting GPT prompts for ~70% token savings

Check out the accompanying blog post here.

Installation

$ pip install compress-gpt

Usage

Simply change your existing imports of langchain.PromptTemplate to compress_gpt.langchain.CompressTemplate (to compress prompts before populating variables) or compress_gpt.langchain.CompressPrompt (to compress prompts after populating variables).

-from langchain import PromptTemplate
+from compress_gpt.langchain import CompressPrompt as PromptTemplate

For very simple prompts, use CompressSimplePrompt and CompressSimpleTemplate instead.

If compression ever fails or results in extra tokens, the original prompt will be used. Each compression result is aggressively cached, but the first run can take a hot sec.

Clearing the cache

import compress_gpt

compress_gpt.clear_cache()

Demo

How CompressGPT Works

My blog post helps explain the below image.