llmux is an operating system on Large Language Models that provides an abstraction layer with basic funtionalities such as multiagent concurrency and communication, long-term memomy management, device management, etc. It is designed to help building complex applications on large language model chatbots. llmux is currently an experimental project under rapid development. We encourage you to build applications and run experiments on llmux. You can share your user experience, report issues, or start pull requests to help us improve llmux. Your contributions will be greatly appreciated.
pip install -r requirements.txt
You may read the technical note here.
First set up the environment variable OPENAI_API_KEY. Run python3 demo.py for a demo. This script illustrates the basics of llmux. You may read and modify it according to your needs. You can start your message with "! ' to evaluate an expression. Call ! system.exit() to exit gracefully. You can only see messages sent to you on the terminal. To inspect all messages, such as messages between a bot and the system or messages between bots, inspect the output files. By default the output path is "/home/ubuntu/llmux_output", you can modify this in "demo.py"
@misc{llmux,
title={An Operating System on Large Language Models for General-Purpose Lifelong Learning},
author={Hangrui Bi},
year={2023}
}
- More testing and prompt engineering.
- Improve the regex used for extracting codes in 'bot.py'.
- Improve the format error messages, distinguish error types, identify unsuccessful function calls and warn.
- Implemenet more devices.
- Add an event handler to write old messages into the long-term memory.
- Implement advanced memory management, compression and garbage collector.
- Load balance among multiple backends.
- Allow event handlers to filter messages within a peer.
- Support large long-term device that does not fit in RAM.