Deep learning

• Demonstrate knowledge of machine learning terminology such as likelihood function, maximum likelihood, Bayesian inference, feed-forward, convolutional, sequential and Transformer neural networks, and error back propagation.
• Understand and explain the choices and limitations of a model for a given setting.
• Apply and analyze results from deep learning models in exercises and own project work*.
• Plan, delimit and carry out an applied or methods-oriented project in collaboration with fellow students and project supervisor*.
• Assess and summarize the project results in relation to aims, methods and available data*.
• Carry out the project and interpret results by use of computational framework for GPU programming such as PyTorch*.
• Structure and write a final short technical report including problem formulation, description of methods, experiments, evaluation and conclusion*.
• Organize and present project results at the final project presentation and in report*.
• Read, evaluate and give feedback to work of other students.
• * If generative AI is used in these phases, then it needs to be documented and critically assessed. A checklist will be provided and should be handed in as part of the report.

Course outline week 1-8:
1. Introduction to statistical machine learning, feed-forward neural networks (FFNN) and error back-propagation. Part I do it yourself on pen and paper.
2. Introduction to statistical machine learning, feed-forward neural networks (FFNN) and error back-propagation. Part II do it yourself in NumPy.
3. Introduction to statistical machine learning, feed-forward neural networks (FFNN) and error back-propagation. Part III PyTorch.
4. Convolutional neural networks (CNN) + presentation of student projects.
5. Sequence modelling for text data with Transformers.
6. Tricks of the trade and data science with PyTorch + Start of student projects.
7. Variational learning and generative adversarial networks for unsupervised and semi-supervised learning.
8. Reinforcement learning – policy gradient and deep Q-learning.

Starting from week 6 and full time from week 9 and the rest of the term will be spent on supervised project work.

02450/01005/02405/02402/02403/02409/02002/02631/02632/02633/02634, Calculus (chain rule of differentiation), basic linear algebra, basic multivariate probability theory, statistics and machine learning (maximum likelihood, Bayes, over- and underfitting, regularization) and programming in Python.

Flipped classroom (video lectures), exercises, mandatory assignments with peer assessment, and student project (1-5 in each group).

Deep learning-based machine learning is behind many recent advances in generative AI, speech recognition, image classification, and reinforcement learning. These developments have been mainly driven by the availability of fast computers, an abundance of data and algorithmic improvements for training neural networks. In the coming years, these methods can lead to technical advancements in many data-driven areas.

Deep learning and machine learning generally see a lot of activity both within academia and industry. This course will provide the student with the foundation to apply deep learning in research and industry.

This is a machine learning course with a focus on computation and application. The project will be carried out with researchers at DTU or in a machine learning-based company.

The course is a part of the focus area Machine Learning and Signal Processing of the Master of Mathematical Modelling and Computing program.

Minimum: 20.

Please be aware that this course will only be held if the required minimum number of participants is met. You will be informed 8 days before the start of the course, whether the course will be held.