Understanding Perceptron training algorithm

1.8k views Asked by At

The following text is from Hal Daumé III's "A Course in Machine Learning" online text book (Page-41).

enter image description here

I understand that, D = size of the input vector.

(1) What kind of Perceptron algorithm is this? Binary/Multi-class? Online/Offline?

(2) What is y here? Bias/weight/Sample/class_label?

(3) What is the rationale of testing ya<=0 for updating weights?


EDIT.

y is class_label.

1

There are 1 answers

0
ibezito On BEST ANSWER

Answers to your questions:

1 - This is a binary perceptron algorithm, working on an offline batch.

2 - as you wrote - Y is the labels vector. each label can be either be 1 or -1.

3 - The rational of testing if y*a<=0 is to check if the perceptron classified a certain sample correctly. If not - the weights of the perceptron are modified.

A bit more about the 3rd question

The idea behind the perceptron algorithm is as follows:

a. we iterate over the samples MaxIter times.

b. The perceptron classifies each sample by multiplying it with the weights vector W and adding a bias b. The result is assigned into the variable a.

c. The prediction for each sample can be either 1 or -1. It is calculated by sign(a). At this stage we check the correctness of the classification.

if y*a>0 that means that y=sign(a). In other words the predicted classification is correct, and we move on to the next sample.

If however y*a<=0, that means that the perceptron failed to predict the correct label. In this case, the algorithm changes the perceptron's weights in a way that they'll be more compatible to the sample which we failed to classify.