cog-isa / htm-rl

HTM applied to RL

This repository has been deprecated in favor of the him-agent.

---

HTM applied to RL setting

• HTM applied to RL setting
  • Links
  • Quick install
  • Repository structure
  • Working example
    • How to run HIMA agent
    • Non-HIMA Q-learning agent

Links

• This repo link
• Contributors guide
• Cumulative project's readme
• Introductory materials

Quick install

There're two setup guides:

• quick & short version is here. It's recommended for htm_rl library users.
• extended version is for contributors or if you have troubles with this version.

# create env with required packages via conda, then activate it
conda create --name htm python=3.9 numpy matplotlib jupyterlab ruamel.yaml tqdm wandb mock imageio seaborn
conda activate htm

# install packages, that cannot be installed with conda, with pip
pip install hexy prettytable pytest>=4.6.5

# git clone our `htm.core` fork to an arbitrary place and pip install it from sources
# pip will install missing dependencies to the current environment if needed
cd <where to clone>
git clone https://github.com/ZhekaHauska/htm.core.git
cd htm.core
pip install --use-feature=in-tree-build .

#  cd to the htm_rl subdirectory in project root and install htm_rl package
cd <htm_rl_project_root>/htm_rl
pip install -e .

Repository structure

• notebooks/ - Jupyter Notebooks
• reports/ - any [markdown, tex, Jupyter Notebooks] reports
• tools/ - any 3rd party tools and scripts
• watcher/ - visualization tool for HTM SP and TM.
• htm_rl/ - sources root (mark this directory in for PyCharm), it contains setup.py
  • htm_rl/ - htm_rl package sources
    • run_X.py - runners, i.e. entry point to run testing scenarios

Working example

How to run HIMA agent

See corresponding readme

Non-HIMA Q-learning agent

# cd to package sources root
cd <htm_rl_project_root>/htm_rl/htm_rl/

# cd to the 5x5_pos experiments
cd experiments/5x5_pos/

# runs random agent and Q-learning agent with learned model 
# on 5x5 env with an agent position as the observation
python ../../run_experiment.py -c debug -e pos -a rnd qmb