Single-Course Engelsk 5 ECTS

Optimization and Data Fitting

Overall Course Objectives

An engineer is often faced with the problem of having to determine optimal values of the parameters in a mathematical model of a physical or technical problem. The problem is eg to find the parameters in a function so that the corresponding curve is a best fit to a given set of data points, or you may be given a mathematical formula that expresses the cost of producing a commodity or perform a transportation job. Here you have to choose values for the free parameters so that the cost is minimized.
The course deals with efficient methods for computing optimal values for the parameters in a mathematical model. The students will study and use available software libraries and learn how to construct their own programs.

See course description in Danish

describe basic concepts in continuous optimization: gradient, Hessian, convexity, descent directions and methods, optimality conditions
explain basic methods for unconstrained optimization, eg the steepest descent and Newton’s methods
explain the basic design paradigms for optimization algorithms: line search and trust regions
implement simple optimization algorithms in Python
apply existing Python programs to the solution of a given problem
formulate a mathematical model to use in data fitting
choose between alternative methods for determining the model parameters: least squares, L1, Huber estimation, and other regression methods
apply and implement Newton and Quasi-Newton methods for unconstrained optimization
implement derivative-free methods
apply conjugate gradient methods for large-scale unconstrained optimization
derive and use the KKT optimality condition for constrained optimization
apply surrogate methods for non smooth optimization

Methods for finding minimum points of a smooth function (e.g. steepest descent, Newton’s and quasi-Newton methods, trust regions, conjugate gradient). Special methods for least-squares data fitting (e.g., the Levenberg-Marquardt algorithm) and non smooth problems. The course content is taught using technical case studies and available numerical libraries.

02601/02402/01020, Introductory numerical algorithms, introductory statistics, and experience with programming in Python. Coursework in linear algebra (e.g. 01020) is recommended.

Lectures and project work.