This course covers the learning outcomes and goals of understanding gradient descent and stochastic gradient descent in deep neural networks. Students will learn about the effects of different learning rates, convergence rates for convex functions, challenges faced in gradient descent, and the implementation of stochastic gradient descent. The teaching method involves lectures and theoretical discussions. The intended audience for this course is individuals interested in deep learning, neural networks, and optimization algorithms.
Overview
Syllabus
Introduction
Gradient Descent Convergence
Recovery Theorem
Proof
Interpretation
Gradient Descent Challenges
Stochastic Gradient Descent
Step sizes and learning rates
Challenges
Learning Rates
Taught by
Paul Hand
Related Courses
-
Deep Learning - IIT Ropar
-
Distributed Optimization and Machine Learning
-
Improving Deep Neural Networks: Hyperparameter Tuning, Regularization and Optimization
5.0 -
Practical Deep Learning for Coders - Full Course
-
Large Scale Machine Learning and Convex Optimization - Lecture 3
-
Stochastic Gradient Descent
