Course information

This course focuses on the principles of optimization, including: Convex Sets and Convex Functions: Introduction to convexity concepts. Fréchet and Directional Derivatives: Mathematical tools and applications. Extrema of Functions: Techniques to find maxima and minima. Existence and Uniqueness Theorems: Fundamental theorems of optimization. Necessary and Sufficient Conditions for Optimality: Detailed study of Lagrange and Kuhn-Tucker-Lagrange multiplier theorems. Quadratic Functions and Least-Squares Methods: Applications of quadratic functions and solving least-squares problems. Optimization Methods: Golden Section Method. Gradient and Conjugate Gradient Methods. Newton-Raphson and Quasi-Newton-Raphson Type Methods. Frank-Wolfe and Projected Gradient Methods. Gradient-Penalty Methods. Penalty and Mixed Gradient-Penalty Methods. Topics in Advanced Optimization: Trust Region Methods and Derivative-Free Optimization Methods.