This course aims to teach learners about building machine learning infrastructure. The learning outcomes include understanding the research and engineering problems involved in ML infrastructure, expressing input pipelines, and optimizing storage and access. The course covers skills such as working with computational graphs, vectorized normalization, and executing ML programs. The teaching method involves a mix of theoretical concepts, case studies, and practical examples. This course is intended for individuals interested in machine learning infrastructure and those looking to enhance their skills in building ML systems.
Overview
Syllabus
Intro
Big data and ML infra are similar
Speaker background
Why invest in ML infra?
Case study: Building a new TF runtime
ML program as a computational graph
An example ML program
Lifetime of an ML program
Vectorized normalization
A slight digression on Eager execution
ML infra and SQL query processing
(Random) scan-based access patterns
Beyond pure dataflow
ML and DB terminology mapping
Recall graph processing workflow
Expressing input pipelines
Decoupled API and execution
Challenge: Randomized transformations
Graph rewrites
Cost model and data stats
Constraint propagation
Storage/access optimizations
Push vs pull based execution
Distributed and parallel execution
ML infra is like data infra, with new twists
Let's collaborate
Taught by
TensorFlow
