Single-Course English 5 ECTS

Stochastic Processes – Probability 2

Overall Course Objectives

(Generally) To learn to formulate and analyse relatively simple dynamic probabilistic models. (Especially) To be acquainted with some models of this type which have proved practically usable.

Learning Objectives

  • Differentiate between different types of stochastic processes, and determine which model class that is most relevant for a certain dynamic phenomenon
  • Simulate realizations of a Markov- or renewal process
  • Classify states of a Markov process and the process itself
  • Determine invariant distributions in Markov processes
  • Determine simple time varying transition probabilities in Markov processes
  • Formulate and solve equations for time to absorption or expected time to absorption in Markov chains.
  • Formulate discrete time Markov processes, which arise from different sampling techniques in continuous time processes
  • Identify and analyze important special cases of Markov processes, e.g. birth and death processes and fundamental queueing systems
  • Perform calculations in models based on Brownian motion
  • Working knowledge of different probability generatingfunctions
  • If time allows get some knowledge on martingales

Course Content

The topics are: Markov chains in discrete and continous time, renewal and Markov renewal processes.

Recommended prerequisites

02405/02402/02403, Basic knowledge of probability and statistics. Distribution function, moments. (E.g. 02402/02403+02405)
Knowledge of programming, e.g. Matlab.

Teaching Method

Lecture, exercises and computer work.



The course is a good supplement for courses in statistics, operations research, time series analysis, image analysis, and control theory.

See course in the course database.





13 weeks




DTU Lyngby Campus

Course code 02407
Course type Candidate
Semester start Week 35
Semester end Week 48
Days Tues 8-12

7.500,00 DKK