This workshop on the Theory of Deep Learning aims to teach participants how to fix GAN optimization through competitive gradient descent. The course covers topics such as single agent optimization, strategic equilibria, applications in machine learning, and competitive gradient descent. The teaching method involves a mix of theoretical concepts and practical applications. This course is intended for individuals interested in deep learning theory and GAN optimization.
Fixing GAN Optimization Through Competitive Gradient Descent - Anima Anandkumar
Institute for Advanced Study via YouTube
Overview
Syllabus
Intro
Single Agent Optimization
Competitive Optimization
Strategic Equilibria
Applications in ML
Alternating Gradient Descent
A polemic
Recall Gradient Descent
How to linearize a game?
Linear or Multilinear?
Competitive Gradient Descent
Why bilinear makes sense
What I think that they think that I think...
Comparison to existing methods
What is the solution of a GAN
Modeling competing agents
The global game
The myopic game
The predictive game
Numerical results
Taught by
Institute for Advanced Study
Related Courses
-
Improving Deep Neural Networks: Hyperparameter Tuning, Regularization and Optimization
5.0 -
Build Basic Generative Adversarial Networks (GANs)
-
Gradient Descent and Stochastic Gradient Descent
-
Analyzing Optimization and Generalization in Deep Learning via Dynamics of Gradient Descent
-
Theory of GANs for Compressed Sensing
-
Stanford Seminar - Deep Learning in Speech Recognition
