Optimization and Data Fitting

• describe basic concepts in continuous optimization: gradient, Hessian, convexity, descent directions and methods, optimality conditions
• explain basic methods for unconstrained optimization, eg the steepest descent and Newton’s methods
• explain the basic design paradigms for optimization algorithms: line search and trust regions
• implement simple optimization algorithms in Matlab
• apply existing Matlab programs to the solution of a given problem
• formulate a mathematical model to use in data fitting
• choose between alternative methods for determining the model parameters: least squares, L1, Huber estimation, and other regression methods
• use optimization to estimate parameters in mathematical models
• apply and implement Newton and Quasi-Newton metoder for unconstrained optimization
• implement line search and trust region algorithms
• implement derivative-free methods
• apply conjugate gradient methods for large-scale unconstrained optimization

Methods for finding minimum points of a smooth function (eg steepest descent, Newton’s and quasi-Newton methods). Special methods for least-squares approximation (e.g., the Levenberg-Marquardt algorithm) and minimax approximation. The course content is taught using technical case studies and available numerical libraries.

02601/02402/02002/02631/02632/02633, Introductory numerical algorithms, introductory statistics, and experience with programming in Matlab (or Python).

Lectures and project work.

Prerequisite for course 02612 “Constrained Optimization”