![Logo](https://private-user-images.githubusercontent.com/891585/254838850-521df7cc-2675-41ba-ac5d-8ca57f92261b.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjIwNzYxMTUsIm5iZiI6MTcyMjA3NTgxNSwicGF0aCI6Ii84OTE1ODUvMjU0ODM4ODUwLTUyMWRmN2NjLTI2NzUtNDFiYS1hYzVkLThjYTU3ZjkyMjYxYi5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwNzI3JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDcyN1QxMDIzMzVaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT1lMjhlZGIyNmI0ZjgwN2I3ZjNjNDJmNDI0NGYyMDI3NDMxMGViMmE0YmMxZjZkOWY3MmM1ZTFjNDQ3YzZhY2QxJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.ZOW_yASuxzYZsbMWAEO7CjBt57ujl1qBQiGla7toSgc)
LLFn (read: 🐘 Elephant) is a light-weight framework for creating applications using LLMs.
Build anything from a simple text-summarizer to complex AI agents with one common primitive: function.
Core Concept · Installation · Examples · Report Bug · Request Feature
LLFn has only one goal: making it 100x easier to experiment and ship AI applications using the LLMs with minimum boilerplate. Even if you're new to LLMs, you can still get started and be productive in minutes, instead of spending hours learning how to use the framework.
- 🔋 Functionify: turning any
prompt
into a callable function. - 📤 Flexible Output: defining LLM result with Python and
pydantic
types. - 🧱 Infinitely Composable: any function can call any other function in any order.
- 🛒 Use Any LLM: leveraging
LangChain
's LLM interface, drop in your favorite LLM with 100% compatibility. - 🪶 Light-Weight: small core framework of LLFn making it extremely easy to understand and extend.
We draw a lot of inspiration from both FastAPI and LangChain, so if you're familiar with either of them you'd feel right at home with LLFn.
The primary goal of LLFn is to encapsulate everything you do with LLMs into a function. Each function comprises of 3 components: input
, prompt
, and output
. Here's how you can create a simple text translator using LLFn.
from llfn import LLFn
function_prompt = LLFn()
@function_prompt
def translate(text: str, output_language: str) -> str:
return f"Translate the following text into {output_language} language: {text}"
To execute this you have to bind
the function to an LLM. You can bind any LangChain's supported language models to LLFn functions.
from langchain.chat_models import ChatOpenAI
# Setup LLM and bind it to the function
llm = ChatOpenAI(temperature=0.7, openai_api_key=API_KEY)
translate.bind(llm)
If you don't want to repeat yourself binding all the functions individually, you can bind directly to function_prompt
as well.
# Alternatively: bind the llm to all functions
function_prompt.bind(llm)
Now you can call your function like you would for any Python function.
translate("Hello welcome. How are you?", "Thai")
# สวัสดี ยินดีต้อนรับ สบายดีไหม?
The beauty of this construct is that you're able to infinitely compose your applications. LLFn does not make any assumption on how you'd chain up your application logic. The only requirement for a function_prompt
is that it returns a prompt string at the end.
And ... that's it! That's just about everything you need to learn about LLFn to start building AI apps with it.
👆 Click to see the full code
import os
from llfn import LLFn
from langchain.chat_models import ChatOpenAI
llm = ChatOpenAI(
temperature=0.7,
model=os.getenv("OPENAI_MODEL"),
openai_api_key=os.getenv("OPENAI_API_KEY"),
)
function_prompt = LLFn()
function_prompt.bind(llm)
@function_prompt
def translate(text: str, output_language: str) -> str:
return f"Translate the following text into {output_language} language: {text}"
# Call the function
print(translate("Hello welcome. How are you?", "Thai"))
LLFn is available on PyPi. You can install LLFn using your favorite Python package management software.
$ pip install llfn # If you use pip
$ poetry add llfn # If you use poetry
While LLFn is simple and light-weight by design, it does pack some punches above its weight that helps you write AI applications more intuitively.
You can create a function to automatically format the response into a desired pydantic
object. To do this, you simply declare the output of your function in pydantic
, and let the LLFn do the heavy lifting for you.
from pydantic import BaseModel, Field
class AgentLogicOutput(BaseModel):
tasks: List[str] = Field(..., description="list of tasks to accomplish the given goal")
reasoning: str = Field(..., description="why you choose to do these tasks")
@function_prompt
def agent_think(goal: str) -> AgentLogicOutput:
return f"You're an agent planning 5 tasks to accomplish the following goal: {goal}"
agent_think("Creating online business") # this returns AgentLogicOutput object
This allows for powerful composability like tools, agents, etc. without sacrificing the simplicity and verbosity of our program. Under the hood LLFn injects the output format into your prompt and automatically parse the LLM result into the specified format.
LLFn let you use both LangChain's LLMs and Chat Models to power your functions. There are 2 ways to assign it.
This is the most convenient way to just get your function to work.
from llfn import LLFn
from langchain.chat_models import ChatOpenAI
llm = ChatOpenAI(temperature=0.7, openai_api_key=API_KEY)
function_prompt = LLFn()
function_prompt.bind(llm)
@function_prompt
def your_function(...):
...
If you want to use different LLMs for different tasks, you can do so by binding llm
to your function individually.
from langchain.llms import LlamaCpp
llm2 = LlamaCpp(...)
your_function.bind(llm2) # This will override the LLM for this function
With LLFn, it's trivial to define callbacks in between execution. Because everything is a function, you can print
or call any 3rd party APIs anywhere you want.
For example, if you want to print results of executions in between a series of complex AI agent it would look like this.
...
# Assuming that other functions are already defined with function_prompt
@function_prompt
def agent_plan_and_execute(goal: str) -> str:
tasks = agent_think(goal)
print(tasks) # Debug
results = agent_execute_tasks(tasks)
print(results) # Debug
evaluation = agent_evaluate_perf(goal, results)
print(evaluation) # Debug
Your function can also take a callback function as a parameter. LLFn does not dictate how you should do it. Anything that works for you would do!
Because LLFn leverages LangChain's LLMs, you can use LangChain Streaming directly.
from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler
llm = ChatOpenAI(
streaming=True,
callbacks=[StreamingStdOutCallbackHandler()],
...
)
function_prompt.bind(llm)
Task | Showcase |
---|---|
Machine Translation | Using LangChain Chat Model |
Machine Translation | Using LangChain LLM Model |
Agent Task Planner | Defining Structured Output |
Task | Showcase |
---|---|
BabyAGI | Implementing BabyAGI with LLFn |
We currently don't take any monetary donations! However, every issue filed and PR are extremely important to us. Here is the roster of contributors and supporters of the project.
Contributions, feedback, and suggestions are highly encouraged and appreciated! You can contribute to LLFn in the following ways:
- 🔀 Fork the repository and make modifications in your own branch. Then, submit a pull request (PR) with your changes.
- ⬇️ Submit issues (GitHub Issues) to report bugs, suggest new features, or ask questions.
If you wish to cite LLFn in your research, we encourage the use of CITATION.cff provided for appropriate citation formatting. For more details on the citation file format, please visit the Citation File Format website.
LLFn is an open-source software under the MIT license. This license allows use, modification, and distribution of the software. For complete details, please see the LICENSE file in this repository.
We would like to express our gratitude to the following projects and communities for their inspiring work and valuable contributions: