The tutorial is aimed at computer science audiences who are a bit unfamiliar with Optimization Techniques.
Some of the popular concepts that we touch upon are as follows:
- Gradient Descent
- Steepest Descent
- Newton's Method
- Quasi-Newton Methods (like BFGS)
- Line Search
- Adam
- RMSprop
- Trust Region
- Interior Point Methods (Linear/Quadratic Programming)
- Simplex Method
- Branch and Bound & Branch and Cut
- Barrier Function
- Non-convex optimization
- Constrained optimization
- Gradient free optimization
- Backpropagation