This is a problem of fitting a Polynomial function for finding a one-dimentional function.
d is the degree of the polynomial.
We need to find a theta that minimize the risk:
Use Problem1.mat file as the dataset which consists of input variable X and output Y.
Split the dataset randomly into training and test sets (you need to adjust some parameters for other datasets).
Cross-validation has been used for finding the optimum order of the polynomial function.
Result:
By increasing the order of polynomial function training loss will decrease significantly and our model will adjust to training set almost perfect. However, as the orders increase our model performance on test set will not be stable and at some point, not only loss function will not improve but also will start increasing again. The point in which the minimum loss results for test set is the critical point between underfitting and overfitting of the system. In this case, the order of D = 9 is our critical point.
