Kohnonen Neural Nets ( Self Organizing Networks )

Posted by ljmacphee on March 9, 2007 under neural networks, source code, topics in artificial intelligence | Be the First to Comment

Kohnonen Neural Nets (Self Organizing Networks)

The Kohonen Self Organizing Map (Network) uses unsupervised, competitive learning. These networks are used for data clustering as in, speech recognition and handwriting recognition. They are also used for sparsely distributed data. Self Organizing Networks consist of two layers, an input layer and a Kohonen layer.The input layer has a node for each dimension of the input vector. The input nodes distribute the input pattern to each node in the the Kohonen layer so the two layers are fully connected. The output layer has at least as many nodes as categories to be recognized. One neurode in the output layer will be activated for each pattern. Each input is connected to each output and there are no connections between the layers.

The network uses lateral inhibition, which is how the vision system works in people. Connections are formed to neighboring neurodes which are inhibitory. The strength of the neurode is inversely proportional to the distance it is away from other nodes. The neurode with the strongest signal dampens the neurodes close to it using a Mexican Hat function. (so called because it looks like a Mexican hat.) The Mexican Hat function is also used in wavelets and image processing..

The neurodes close to the one activated take part in the training, the others do not. To make it computationally efficient a step function is used instead of a true Mexican hat function.

General algorithm
The weights between the nodes are initialized to random values between 0.0 and 1.0.
Then the weight vector is normalized.
The learning rate is set between 1.0 and 0.0 and decreased linearly each iteration.
The neighborhood size is set and decreased linearly each iteration The input vector is normalized and fed into the network.
The input vector is multiplied by the connection weights and the total is accumulated by the Kohonen network nodes.

The winning nodes out put is set to one and all the other nodes are set to zero.
Weights are adjusted Wnew = Wold + training constant ( input Wold) Training continues until a winning node vector meets some minimum error standard.
SelfOrganizing Network example

Neural Net Meshes

Posted by ljmacphee on March 8, 2007 under neural networks, topics in artificial intelligence | Be the First to Comment

Neural Net Meshes

Meshes are used in visualization, image processing, neurology and physics applications. They are a grid of regular or irregular shape that stores information or represents a shape rather than a flat object. Neural nets are used to adjust the meshes in 3d graphics.

Meshes also derived from Pask’s Conversation Theory. The gist of the meshes being that distributed information (like that of the Internet) adapts to the semantic expectations of the users. The system then self organizes to meet expectations.

Counterpropagation Network

Posted by ljmacphee on March 7, 2007 under neural networks, topics in artificial intelligence | 2 Comments to Read

Counterpropagation Network

The counterpropagation network is a hybrid network. It consists of an outstar network and a competitive filter network. It was developed in 1986 by Robert Hecht-Nielsen. It is guaranteed to find the correct weights, unlike regular back propagation networks that can become trapped in local minimums during training.

The input layer neurodes connect to each neurode in the hidden layer. The hidden layer is a Kohonen network which categorizes the pattern that was input. The output layer is an outstar array which reproduces the correct output pattern for the category.

Training is done in two stages. The hidden layer is first taught to categorize the patterns and the weights are then fixed for that layer. Then the output layer is trained. Each pattern that will be input needs a unique node in the hidden layer, which is often too large to work on real world problems.