neural network for the dataset make_moon of scikit-learn.
the numbers above is the answer for each part.
- Build a model with a 3-dimensional hidden layer and plot the decision boundary.
- In the above we picked a hidden layer size of 3. Let’s now get a sense of how varying the hidden layer size (1,2,3,4,5,20,40) affects the result. For each hidden layer size, please plot the decision boundary.
- Instead of batch gradient descent, use minibatch gradient descent (more info) to train the network. Minibatch gradient descent typically performs better in practice.
- We used a fixed learning rate for gradient descent. Implement an annealing schedule for the gradient descent learning rate.
- We used a tanh activation function for our hidden layer. Experiment with other activation functions (some are mentioned above). Note that changing the activation function also means changing the backpropagation derivative.
- Extend the network from two to three classes. You will need to generate an appropriate dataset for this.
- Extend the network to four layers. Experiment with the layer size. Adding another hidden layer means you will need to adjust both the forward propagation as well as the backpropagation code.
I put the full analysis but it is in persian.