HTM 2D Object Modeling
See the origin of this project on this HTM Forum thread.
The purpose of this project is to define 2D Object Recognition Tests.
Current project status
Github is used for the tracking of tasks. See repository Issues.
Task definition
Object Schema
Objects exist within a 2D space. The space has a width and height. Each location in the space can be identified with an X and Y integer coordinate. Each location may have a Feature.
Objects can be defined as collections of Features in 2D space. Here is an example object schema containing one feature X
at 14,5
---
name: Some Object
width: 20
height: 20
features:
- { x: 14, y: 5, data: X }
Features
Features always contain data. A Feature's data is used by the Agent to identify objects.
Initially, all Features consist of a simple data type, but should be extensible to contain any data type. For example, current the current Object Library consists of simple one-character features.
Object Library
The Object Library can be found in objects/
. Each YAML file within this directory contains one object definition in the format specified in "Object Schema" above.
Agency
An Agent can exist in a location within an Object space. An Agent observing an object will receive features in space according to the location of its sensors.
Each Agent has exactly 4 sensors:
- North (Agent Y - 1)
- South (Agent Y + 1)
- East (Agent X + 1)
- West (Agent X - 1)
At one time step, an Agent can be at only one location in Object space. Each sensor has access to the Feature beneath it. Agents should use their sensors to attempt to identify the object under observation at each time step.
Algorithm concept
We need to build a 3-layer Network for each sensor which has an object pooling layer as described in the Columns Paper
above a 2-layer location/sensor circuit as described in Columns+:
Object layers must share representations between cortical columns via lateral connections:
For code examples, see the supporting paper for the Columns paper
Project structure
Visualization of the objects from the Object Library
See javascript readme.
Java
This folder now contains just "Hello World!" program in Java. Code here if you want to use Java language.
Python
Currently used language for the development. See python readme for instructions how to run.