Single-Course English 5 ECTS

Optimization and Data Fitting

Overall Course Objectives

An engineer is often faced with the problem of having to determine optimal values of the parameters in a mathematical model of a physical or technical problem. The problem is eg to find the parameters in a function so that the corresponding curve is a best fit to a given set of data points, or you may be given a mathematical formula that expresses the cost of producing a commodity or perform a transportation job. Here you have to choose values for the free parameters so that the cost is minimized.
The course deals with efficient methods for computing optimal values for the parameters in a mathematical model. The students will study and use available software libraries and learn how to construct their own programs.

Learning Objectives

  • describe basic concepts in continuous optimization: gradient, Hessian, convexity, descent directions and methods, optimality conditions
  • explain basic methods for unconstrained optimization, eg the steepest descent and Newton’s methods
  • explain the basic design paradigms for optimization algorithms: line search and trust regions
  • implement simple optimization algorithms in Matlab
  • apply existing Matlab programs to the solution of a given problem
  • formulate a mathematical model to use in data fitting
  • choose between alternative methods for determining the model parameters: least squares, L1, Huber estimation, and other regression methods
  • use optimization to estimate parameters in mathematical models
  • apply and implement Newton and Quasi-Newton metoder for unconstrained optimization
  • implement line search and trust region algorithms
  • implement derivative-free methods
  • apply conjugate gradient methods for large-scale unconstrained optimization

Course Content

Methods for finding minimum points of a smooth function (eg steepest descent, Newton’s and quasi-Newton methods). Special methods for least-squares approximation (e.g., the Levenberg-Marquardt algorithm) and minimax approximation. The course content is taught using technical case studies and available numerical libraries.

Recommended prerequisites

02601/02402/02002/02631/02632/02633, Introductory numerical algorithms, introductory statistics, and experience with programming in Matlab (or Python).

Teaching Method

Lectures and project work.


Prerequisite for course 02612 “Constrained Optimization”

See course in the course database.





13 weeks




DTU Lyngby Campus

Course code 02610
Course type Bachelor
Semester start Week 35
Semester end Week 48
Days Mon 13-17

7.500,00 DKK