timlrx/simple-ai-agents

Simple AI Agents

Create simple multi-agent workflows using any LLMs - easy to experiment, use and deploy.

The package extends Simple AI Chat by adding support for 100+ LLM providers, structured responses and multiple agents, similar to Autogen. With out of the box handling of LLM requests, session handling and structured response generation, multi-agent conversations can be easily orchestrated with Python to manage the control flow. This results in code which is easy to understand and extend with minimal dependencies.

Note: The package is in active development and the API is subject to change. To use, please clone the repo and install the package locally. It has not been published to PyPI yet. Better JSON support for other LLM models and the ability to save and load sessions are in the roadmap.

Features

Mix and match LLM providers (OpenAI, Huggingface, Ollama, Anthropic and more!).
Create and run chats with only a few lines of code!
Integrates with instructor to provide structured responses for almost all models.
Run multiple independent chats at once or create Autogen like multi-agent conversations.
Minimal codebase: no code dives to figure out what's going on under the hood needed!
Async and streaming.
Interactive CLI.

Getting Started

Add the necessary environment variables corresponding to the LLM providers that will be used. Refer to .env.example for an example of the variables to add:

OPENAI_API_KEY=<your openai api key>
OPENAI_ORGANIZATION=<your openai organization>
HUGGINGFACE_API_KEY=<your huggingfacehub api token>

The package provides a ChatAgent class that can be used to create a chatbot:

from simple_ai_agents.chat_agent import ChatAgent

chatbot = ChatAgent(system="You are a helpful assistant")
chatbot("Generate 2 random numbers between 0 to 100", console_output=True)
chatbot("Which of the two numbers is bigger?", console_output=True)

console_output provides a convenient way to print the chatbot's response to the console. By default, the chatbot uses the openai provider. To use a different provider, pass the llm_options argument to the ChatAgent constructor. For example, to use the mistral model from ollama:

from simple_ai_agents.models import LLMOptions

mistral: LLMOptions = {
    "model": "ollama/mistral",
    "temperature": 0.7,
    "api_base": "http://localhost:11434",
}
chatbot = ChatAgent(system="You are a helpful assistant", llm_options=mistral)

The CLI offers an easy way to start a local chatbot session similar to Simple AI Chat or Ollama but with support for almost all LLM providers.

See the examples folder for other use cases.

CLI

Ensure that you have the necessary environment variables set up. Usage:

aichat [OPTIONS] [PROMPT]

The CLI supports the following options:

--prime: Prime the chatbot with a prompt before starting the chat.
--character: The name of the chat agent.
--model: Specify the LLM model e.g. gpt-3.5-turbo, ollama/mistral etc. Uses gpt-3.5-turbo by default.
--temperature: Specify the temperature for the LLM model. Uses 0.7 by default.
--system: System prompt.
--help

Interactive open-ended chat

aichat --prime

Pass in prompts as arguments

Uses a local instance of the mistral model from ollama to summarize the README file.

cat README.md | aichat --model ollama/mistral "Summarize this file"

Looking for an option that is not available? Open an issue or submit a PR!

Structured responses

To generate a structured response, use the gen_model method:

class Person(BaseModel):
    name: str = Field(description="Name of the person")
    age: int = Field(description="Age of the person")

chatbot = ChatAgent(llm_options=openai)
parsed = chatbot.gen_model(
    "Extract `My name is John and I am 18 years old` into JSON",
    response_model=Person
)

The package automatically selects the best mode to generate JSON for a given provider and model:

Provider	Mode	JSON Structure	Quality of Output
Open AI	Tools	Excellent	Excellent
Anyscale	JSON Schema	Excellent	Very good
Ollama	JSON	Excellent	Varies by model
Others	MD_JSON	Acceptable	Varies by model

Currently, Open AI and Anyscale are the only two providers with a structured response mode. Ollama also supports json response but the quality of the output is dependent on the model used (and probably whether it has been fine-tuned for json generation). For other providers, the package uses the MD_JSON mode which is a markdown representation of the JSON structure.

Examples

Development

Poetry

Package management is handled by poetry. Install it by following the instructions here.

Installing packages

After installing poetry, install the project packages by running:

poetry install

Setting up pre-commit hooks

Pre-commit hooks automatically process your code prior to commit, enforcing consistency across the codebase without the need for manual intervention. Currently, these hooks are used:

trailing-whitespace: trims trailing whitespace
requirements-txt-fixer: sorts entries in requirements.txt
black: to format the code consistently
isort: to sort all imports consistently
flake8: as a linter

All commits should pass the hooks above before being pushed.

# Install the configured hooks
poetry run pre-commit install

# Run the hooks on all files in the repo
poetry run pre-commit run -a

If there are any failures, resolve them first, then stage and commit as usual. Committing will automatically trigger these hooks, and the commit will fail if there are unresolved errors.

Run the tests

poetry run pytest

timlrx / simple-ai-agents