🦜🦾 LangGym

A framework for building Natural Language RL Agents.

NOTE ℹ️
This project is currently a proof of concept for a broader framework for using language models as the agent's behavior policy instead of traditional reinforcement learning techniques to power agents.

⚙️ Setup

Create a virtual environment

python -m venv ~/venvs/langgym
source ~/venvs/langgym/bin/activate
pip install -e .

Export your OpenAI API key:

export OPENAI_API_KEY=...

Run a simulation with

python langgym/simple.py --max-cycles 20 --goal-reward -0.05 --output-dir experiments

Visualize simulations with the streamlit dashboard:

streamlit run langgym/dashboard.py

These are a few ideas to improve the NLRLA (natural-language generative agent):

🧠 Memory: Add a text representation of the agent's experiences so far into its internal memory.
📝 Summarized Memories: Store an external memory buffer of observations, actions, and rewards, and ask the agent to summarize its experience so far. Add this to the agent's internal memory.
💭 Strategic Reflection: Ask the agent to create high-level strategies, which are feed into its own prompt when generating an action.

These are items in the roadmap, in no particular order:

👉 Support multi-agent Environments in the Multi Particle Environment suite from the PettingZoo library (see here)
👉 Support additional environment suites, like Atari, Classic, etc.
👉 Create LangGym API for traversing a Universe of different Environments. Agents can choose which environments they want to play in.
👉 Create persistent agents that can store internal memories across their experiences across the different Environments.
👉 Support use of lighter-weight language models like Alpaca, Pythia, etc.