Overview
This course focuses on predicting final network parameters for unseen deep architectures without training them. The learning outcomes include understanding Graph-Hypernetworks, training on diverse DNN-architectures, and predicting high-performing weights. The course teaches skills such as utilizing Graph Neural Networks, message passing, differentiable normalization, and meta-batching. The teaching method involves a lecture-style format with explanations and experimental results. The intended audience for this course includes deep learning enthusiasts, researchers, and practitioners interested in neural architecture search and meta-learning.
Syllabus
- Intro & Overview
- DeepNets-1M Dataset
- How to train the Hypernetwork
- Recap on Graph Neural Networks
- Message Passing mirrors forward and backward propagation
- How to deal with different output shapes
- Differentiable Normalization
- Virtual Residual Edges
- Meta-Batching
- Experimental Results
- Fine-Tuning experiments
- Public reception of the paper
- At , Boris mentions that they train the first variant, yet on closer examination, we decided it's more like the second
Taught by
Yannic Kilcher