Herself’s Artificial Intelligence

Sparse distributed memory first appeared in 1998 as a model of long term memory in humans. The main idea is that distances between concepts in our brains can be represented as distances between points in a high dimension world. Since distances between points are far apart in many dimensions, the distance between concepts is large. The large distance between concepts means I only have to come closer to it than another idea for a match to be made.

For example if I give each letter of the alphabet its own dimension then map it by position in a word then ‘aeple’ is closer to ‘apple’ than ‘ample’ and I can guess that the correct word for the mistyped word. Or a four legged creature that is tall and is spotted is closer to a giraffe than a short legged spotted leopard.

This type of storage of data means you can store far fewer examples you need to match allowing you to store more information in a much smaller memory footprint.

All input is represented in binary form in sparse distributed memory. The algorithm works by calculating the ‘Hamming distance’ between data input and existing examples in memory.

Books:
Reinforcement Learning: An Introduction (pdf download )

More information:
Sparse Distributed Memory
Kevin Kelly ‘Out of Control’ Chapter 2
Sparse Distributed Memory and Related Models Chapter 3 (pdf)

Papers:
Kanerva’s Sparse Distributed Memory An associative memory algorithm well suited to the connection machine. (pdf) ( exellent starting point )
Kanerva’s Sparse Distributed Memory, Multiple Hamming Thresholds ( pdf )