Kernel SVM training with data in a fixed sequence

[Gourment 李大宝]

Hello, I find a very interesting thing: after changing the code of kernel SVM as below, a better training progress and a better performance is shown.

#rand_index = np.random.choice(len(x_vals), size=batch_size)
#rand_x = x_vals[rand_index]
#rand_y = np.transpose([y_vals[rand_index]])
rand_x = x_vals
rand_y = y_vals.reshape(-1, 1)

I think the reason is that every scalar bi in Vector Variable b should follow the ith data pair(feature,label) according to the dual formula. But random choice makes the weight bi be trained by different data pair ,not only the ith and maybe give a wrong direction to update bi.

And tatnguyennguyen give the reason why the current code get a good result.
#104

So the current code is only ok for the whole-batch training. Maybe embedding layer would be used for a mini batch training. :)

[Gourment 李大宝]

SGD can not be used as optimizer of Kernel SVM directly, the mentioned code in the book should be changed.