Optimization and Data Fitting
Overall Course Objectives
An engineer is often faced with the problem of having to determine optimal values of the parameters in a mathematical model of a physical or technical problem. The problem is eg to find the parameters in a function so that the corresponding curve is a best fit to a given set of data points, or you may be given a mathematical formula that expresses the cost of producing a commodity or perform a transportation job. Here you have to choose values for the free parameters so that the cost is minimized.
The course deals with efficient methods for computing optimal values for the parameters in a mathematical model. The students will study and use available software libraries and learn how to construct their own programs.
See course description in Danish
Learning Objectives
- describe basic concepts in continuous optimization: gradient, Hessian, convexity, descent directions and methods, optimality conditions
- explain basic methods for unconstrained optimization, eg the steepest descent and Newton’s methods
- explain the basic design paradigms for optimization algorithms: line search and trust regions
- implement simple optimization algorithms in Python
- apply existing Python programs to the solution of a given problem
- formulate a mathematical model to use in data fitting
- choose between alternative methods for determining the model parameters: least squares, L1, Huber estimation, and other regression methods
- use optimization to estimate parameters in mathematical models
- apply and implement Newton and Quasi-Newton metoder for unconstrained optimization
- implement derivative-free methods
- apply conjugate gradient methods for large-scale unconstrained optimization
- derive and use the KKT optimality condition for constrained optimization
Course Content
Methods for finding minimum points of a smooth function (eg steepest descent, Newton’s and quasi-Newton methods). Special methods for least-squares approximation (e.g., the Levenberg-Marquardt algorithm) and minimax approximation. The course content is taught using technical case studies and available numerical libraries.
Teaching Method
Lectures and project work.
Faculty
Remarks
Prerequisite for course 02612 “Constrained Optimization”