Stochastic Processes – Probability 2

• Differentiate between different types of stochastic processes, and determine which model class that is most relevant for a certain dynamic phenomenon
• Simulate realizations of a Markov- or renewal process
• Classify states of a Markov process and the process itself
• Determine invariant distributions in Markov processes
• Determine simple time varying transition probabilities in Markov processes
• Formulate and solve equations for time to absorption or expected time to absorption in Markov chains.
• Formulate discrete time Markov processes, which arise from different sampling techniques in continuous time processes
• Identify and analyze important special cases of Markov processes, e.g. birth and death processes and fundamental queueing systems
• Perform calculations in models based on Brownian motion
• Working knowledge of different probability generatingfunctions
• If time allows get some knowledge on martingales

The topics are: Markov chains in discrete and continous time, renewal and Markov renewal processes.

02405/02402/02403, Basic knowledge of probability and statistics. Distribution function, moments. (E.g. 02402/02403+02405)
Knowledge of programming, e.g. Matlab.

Lecture, exercises and computer work.

The course is a good supplement for courses in statistics, operations research, time series analysis, image analysis, and control theory.