Gradient-Descent
Gradient-Descent is an algorithm used to find the the minimum of a function. An arbitrary point will move along with opposite direction according to negative of derivative until it arrive to the minimum position.
Step
- Choose an arbitrary point (blue point)
- Calculate the derivative of function on this point
- Calculate the gradient by negative of derivative and a very small learning rate
- Move to a new point according to the gradient
- It will arrive to the minimum position (red point) until maximum iteration or very small change
Example
