I am learning Pattern Recognition and Machine Learning and I was required to write a multiclass perceptron like this:
there are vectors: v_i = [v_i0, v_i1, v_i2, ..., v_im], i = (1, 2, ..., n)
write them to augmented style: x_i = [v_i0, v_i1, v_i2, ..., v_im, 1]
initial value w_i(0) = [0 for _ in range(m)], C
for iteration ITER:
for i in range(n):
step = k = ITER * n + i
use x_i as learning example:
calculate d_j = w_j(k) * x_i, j = (1, 2, ..., n)
if d_i(k) > d_j(k) is TRUE for any (j != i):
w_j(k+1) = w_j(k)
else if exists l such that d_i(k) <= d_l(k):
w_i(k+1) = w_i(k) + C * x_i
w_l(k+1) = w_l(k) - C * x_i
w_j(k+1) = w_j(k) for any (j != i,l)
Then I have a question that:
if there are N vectors w_l(k) such that d_i(k) <= d_l(k)
should I do w_i(k+1) = w_i(k) + C * x_i once only when all comparations between d_i and d_j is over so that w_i(k+1) = w_i(k) + C * x_i
like this:
PUNISH = 0
for _j in range(1,n):
j = (i + _j) % n
if d[i] > d[j]:
COUNT += 1
else:
w[j] -= C * x[i]
PUNISH += 1
if PUNISH > 0:
w[i] += C * x[i]
or I do w_i(k+1) = w_i(k) + C * x_i each time d_i(k) <= d_l(k) so that w_i(k+1) = w_i(k) + N * C * x_i
for _j in range(1,n):
j = (i + _j) % n
if d[i] > d[j]:
COUNT += 1
else:
w[i] += C * x[i]
w[j] -= C * x[i]