Herself’s Artificial Intelligence

Adeline Neural Nets

Adeline, ADAptive linear Neuron was developed by Widrow and Hoff in 1959. It is a classic example of an ‘Adaptive Filter Associative Memory Neural Net’ or ‘Adaptive linear Element’. It has only an input layer consisting of a node for each input and an output layer that has only one node. It can learn to sort linear input into two groups. Inputs are real numbers between -1..+1. The 208 neurode forms a weighted sum of all inputs and output’s a +/-1. There is one input with a weighted synapse for every number in the input vector. It has an extra input ‘mentor’ used during training which carries the expected output for the given input.

Adeline can only separate data in to two groups. The data must be linearly separable. The Adeline’s training starts with a straight line drawn anywhere on the plot provided it intersects the origin. The training effectively rotates this line until it properly separates the data into the two groups, using the least mean squares algorithm. The angle of this line is the angle Adeline tests against the input vector times the weight vector (dot product). If the angle of the dot product of these two vectors is less than a 1 is output, if it is less than a 0 is output.

Dot product:A*B or Ax Bx + Ay By + …. or ABcos() where theta is the angle between vector A and vector B. A bold A or B represents the length of the vector. (now adjust the weights by the amounts in the vector) The learning constant must be less than 2 or the network will not stabilize.

Input patterns are used to set the initial weights, during which time the mentor node is set to +/ 1 depending on the desired output. Following that a training set, different from the initial set, is tried. If the answer is correct we do nothing. If the answer is not correct the weights are adjusted using the delta rule.

The delta rule changes the weights in proportion to the amount they are incorrect. The distance is determined by subtracting network’s actual response difference from expected response; multiply this by a training constant; multiply by the size and direction of the input pattern vector; and use this information to determine the change in weight. This is also known as the Least Mean Squared Rule ChangeInW eight = 2 LearningRate InputN odej (DesiredOutput ActualOutput)

Collections of Adeline’s in a layer can be taught multiple patterns. Adelines can have additional inputs that are powers or multiplications of inputs and are referred to as higher order networks. It may work better at pattern solving than a many layered single order network. This may be used in more than two dimensions. A line separates linear data in a plane, a plane separates linear data in three dimensions, etc. Adelines and Madelines can be used to clean up noise from data provided there is a good copy of the data to learn from during training.