Herself’s Artificial Intelligence

Support Vector Machines

Support Vector Machines (SVM) are based on a non-linear generalization of the ‘Generalized Portrait’ algorithm developed in Russia in the 1960s. They have been around since the 1970s but only recently have begun to attract attention. They have been successfully used for handwriting and speech recognition, as well as speaker recognition and have the ability to pull objects such as faces out of images.

Support vector machines can sort data into two classes, it is in the set or it is not in the set. Data which is not linearly separable can become linearly separable in higher dimensions. However, if data is put into too many dimensions then data classes are memorized rather than learned. If the data is memorized then new data is not handled well, this is known as over-fitting. The error rate of SVM can be explicitly calculated which can not be done for neural networks.

We want to create a hyperplane that gives the maximum possible distance between the points in the set and the points out of the set, with the maximum margin around it. So we want the maximum distance between the point in each set that is closest to the other group. We then create a margin of two lines between the data. The main method used to do this is using Lagrange Multipliers. (aka the ‘Quadratic Programming Problem’ most better spreadsheets and math programs have this built in to them.)

The ‘kernel’ is a formula for the dot product in higher dimensional ‘feature space’. Feature space is the higher dimension space we have mapped our data into to make it linearly separable. A Polynomial of various dimensions and Gaussian Kernels are the most commonly used.

More information:
Overview of Support Vector Machines
Intro to SVM