This conference talk explores how Netflix solved batch ETL challenges using Iceberg and Maestro technologies. Join Jun He, Tech Lead at Netflix, as he demonstrates how to overcome late-arriving data issues while maintaining data accuracy, freshness, and cost efficiency at scale. Dive into the technical architecture that enables incremental change capture through Iceberg's metadata layer without accessing user data, and learn how Maestro orchestrates complex, multi-stage workflows. The presentation includes real-world use cases from Netflix's data engineering practices, providing practical insights for implementing similar solutions in large-scale data analytics environments. The 45-minute talk offers valuable architectural insights for data engineers facing similar challenges in their ETL pipelines.
Overview
Syllabus
Late Data? Netflix's Iceberg + Maestro Solves Batch ETL Chaos
Taught by
InfoQ