Amazon's Exabyte-Scale Migration from Spark to Ray

Explore a detailed case study presentation from All Things Open 2024 examining Amazon's massive 4-year data management migration journey. Dive into the evolution from Oracle to Apache Spark, and ultimately to Ray, as Amazon sought more efficient and scalable open distributed computing frameworks. Learn about the architectural requirements for managing thousands of daily petabyte-scale data processing jobs, discover how Ray deployment on EC2 achieved over 80% cost savings compared to Spark on EMR, and understand Amazon's open-source contributions that extend these benefits to catalog formats like Apache Iceberg.