Introduction to Deep Learning

Overview

This 3-credit-hour, 16-week course covers the fundamentals of deep learning. Students will gain a principled understanding of the motivation, justification, and design considerations of the deep neural network approach to machine learning and will complete hands-on projects using TensorFlow and Keras.

Syllabus

Module 1: Introduction to Deep Feedforward Networks

- Gradient-based learning
- Sigmoidal output units
- Back propagation

Module 2: Regularization for Deep Learning

- Regularization strategies
- Noise injection
- Ensemble methods
- Dropout

Module 3: Optimization for Training Deep Models

- Optimization algorithms: Gradient, Hessian-Free, Newton
- Momentum
- Batch normalization

Module 4: Convolutional Neural Networks

- Convolutional kernels
- Downsampled convolution
- Zero padding
- Backpropagating convolution

Module 5: Recurrent Neural Networks

- Recurrence relationship & recurrent networks
- Long short-term memory (LSTM)
- Back propagation through time (BPTT)
- Gated and simple recurrent units
- Neural Turing machine (NTM)