Scaling Shopify's Search: Enhancing Elasticsearch Resilience With Kubernetes and KEDA

This conference talk explores how Shopify enhanced the resilience of their search infrastructure, which hosts over two petabytes of data serving millions of users worldwide. Learn how the team improved Elasticsearch cluster performance by isolating production writes from maintenance writes that previously degraded search availability. Discover their innovative approach using Kubernetes native mechanisms to host production workloads on isolated Google Cloud nodepools while directing heavy maintenance writes to dedicated autoscalable nodepools. See how implementing Kubernetes-based Event Driven Autoscaling (KEDA) enabled maintenance nodepools to scale only when needed, resulting in 40% cost savings and a 65% improvement in production write performance. Perfect for cloud engineers and architects interested in optimizing large-scale search infrastructure using cloud native technologies.