Class Central is learner-supported. When you buy through links on our site, we may earn an affiliate commission.

RWTH Aachen University

Deep Learning

RWTH Aachen University via edX

Overview

Artificial neural networks form the foundation of modern AI systems. “Deep Learning” offers participants a comprehensive introduction to the core principles and fundamental building blocks used in today’s neural networks. The course covers the most important types of neural networks, like MLPs, CNNs, RNNs, and Transformers, as well as practical techniques for efficient training and the reuse large pre-trained models.

Throughout the course, students will gain a robust understanding of the general training process and key differences between different network types, as well as practical knowledge through hands-on programming exercises.

By the end of the course, students will be equipped with the knowledge and skills to understand, train, and apply deep neural networks to a variety of problems, laying a strong foundation for advanced exploration of the field.

Syllabus

Week 1 – Introduction to Deep Learning

In the first week, we will give an overview of the history of deep learning, covering important milestones and factors of rapid progress that the field has experienced in recent years. In addition to this overview, you will learn about the essentials of neural network training: multi-layer perceptrons, activation functions, error functions, and the backpropagation algorithm.

Week 2 – Practical Deep Learning

In week two, we will have a closer look at some of the important practical aspects of training neural networks. In particular, we will cover data preprocessing and weight initialization, adaptive first-order optimization methods, and some of the typical tricks that enable reliable training and good task performance.

Week 3 - Convolutional Neural Networks

Week 3 introduces Convolutional Neural Networks (CNNs) as a practical way to process image data with neural networks. We will motivate CNNs as parameter-efficient learnable image filters, present typical CNN operators like pooling layers and give an overview of tried and proven architectures for image classification. Additionally, we introduce the encoder-decoder architecture, which is not only a popular pattern to tackle dense image processing tasks, but also a generally useful way to map inputs to outputs.

Week 4 - Recurrent Neural Networks

In week 4, we will present network architectures for natural language processing. In particular, we will introduce Recurrent Neural Networks (RNNs) for sequence processing and analyse their training behaviour, uncovering reasons for unstable training. We will also study Long Short-Term Memory (LSTM) networks, which exhibit more stable training behaviour, and the attention mechanism as a way to improve network architectures for sequence-to-sequence processing.

Week 5 - Transformers

In week 5, we will introduce the fundamental architecture behind modern deep neural networks, the Transformer. Starting from a learnable key-value storage mechanism as a motivating example, we present the basic building blocks of the architecture, covering self- and cross-attention, positional encodings, Transformer Encoder and Decoder, as well as three examples of powerful applications of this versatile architecture.

Week 6 - Large-Scale Learning

In the final week, we will cover large-scale learning systems. Motivated from empirical findings on the benefits of scaling to large models and large amounts of data (so-called scaling laws), we present the typical pretraining and fine-tuning paradigm. Additionally, we show how these large-scale models enable easy construction of multi-modal models from existing architectural building blocks.

Taught by

Prof. Dr. Bastian Leibe and Christian Schmidt M.Sc.

Reviews

Start your review of Deep Learning

Never Stop Learning.

Get personalized course recommendations, track subjects and courses with reminders, and more.

Someone learning on their laptop while sitting on the floor.