Overview
The course teaches a novel algorithm called SynFlow for pruning neural networks without the need for any data. The learning outcomes include understanding the Lottery Ticket Hypothesis, layer collapse in pruning algorithms, and the concept of synaptic saliency conservation. The course aims to equip learners with the skills to iteratively prune neural networks while conserving synaptic flow. The teaching method involves theoretical explanations, algorithm design, and experimental verification. The intended audience for this course includes individuals interested in deep learning, neural networks, and algorithm optimization.
Syllabus
- Intro & Overview
- Pruning Neural Networks
- Lottery Ticket Hypothesis
- Paper Story Overview
- Layer Collapse
- Synaptic Saliency Conservation
- Connecting Layer Collapse & Saliency Conservation
- Iterative Pruning avoids Layer Collapse
- The SynFlow Algorithm
- Experiments
- Conclusion & Comments
Taught by
Yannic Kilcher