Implementations of Triadic Memory and related algorithms in the following programming languages:
Dyadic Memory realizes an associative memory for sparse hypervectors which has the functionality of a Sparse Distributed Memory (SDM) as proposed by Pentti Kanerva in 1988.
The present, highly efficient algorithm was discovered in 2021 and is based on a neural network with combinatorial connectivity.
The memory stores and retrieves heteroassociations x -> y of sparse binary hypervectors x and y.
Sparse binary hypervectors are also known as Sparse Distributed Representations (SDR).
Here x and y are binary vectors of dimensions n1 and n2 and sparse populations p1 and p2, respectively.
While in typical SDM usage scenarios n1 and n2 are equal, the present algorithm also allows asymmetric configurations.
The capacity of a symmetric memory with dimension n and sparse population p is approximately (n/p)^3 / 2.
For typical values n = 1000 and p = 10, about 500,000 associations can be stored and perfectly recalled.
The Dyadic Memory algorithm was initially developed in Mathematica language and consists of just 10 lines of code.
The plain C implementation best illustrates the algorithm in procedural language. This version works with vector dimensions up to n = 1,200.
A memory-optimized implementation supports hypervector dimensions up to n = 20,000. It can be used as a command line tool or as C library. No other SDM currently works with dimensions that large.
An Odin implementation is available here.
A Numba-accelerated Python version is available here.
Triadic Memory, an algorithm developed in 2021, is an associative memory that stores ordered triples of sparse binary hypervectors (also called SDRs).
After storing a triple {x,y,z} in memory, any of the three items can be recalled by specifying the other two parts: {_,y,z} recalls x, {x,_,z} recalls y, and {x,y,_} recalls z. Given three items {x,y,z}, one can test if their association is stored in memory by calculating, for instance, the Hamming distance or overlap between {x,y,_} and z. This remarkable property, absent in hetero-associative memories, makes Triadic Memory suitable for self-supervised machine learning tasks.
A Triadic Memory has the capacity to store (n/p)^3 random triples of hypervectors with dimension n and sparse population p. At a typical sparsity of 1 percent, it can therefore store and perfectly retrieve one million triples.
The original Mathematica code can be found here. The plain C implementation can be compiled as a command line program or as a library. It's also a good starting point for people wanting to port the algorithm to another programming language.
Performance-optimized implementations are available for Python, the Julia language, Chez Scheme, and the Odin.