Class Central is learner-supported. When you buy through links on our site, we may earn an affiliate commission.

CNCF [Cloud Native Computing Foundation]

The State and Future of Cloud-Native Model Serving

CNCF [Cloud Native Computing Foundation] via YouTube

Overview

Explore the state and future of cloud-native model serving in this 39-minute conference talk by Dan Sun from Bloomberg and Theofilos Papapanagiotou from Amazon. Gain insights into KServe, a cloud-native open-source project for serving production ML models built on CNCF projects like Knative and Istio. Learn about KServe's progress towards version 1.0, recent developments such as ModelMesh and InferenceGraph, and its future roadmap. Discover the Kubernetes design patterns used in KServe to achieve core ML inference capability, and understand its design philosophy and integration with the CNCF ecosystem. Examine how the InferenceService interface simplifies networking, lifecycle, and server configurations, enabling easy addition of serverless capabilities to model servers like TensorFlow Serving, TorchServe, and Triton on CPU/GPU. Explore scenarios demonstrating quick KServe implementation and evolution to production-ready setups with scalability, security, observability, and auto-scaling acceleration using CNCF projects such as Knative, Istio, SPIFFE/SPIRE, OpenTelemetry, and Fluid.

Syllabus

The State and Future of Cloud-Native Model Serving - Dan Sun, Bloomberg & Theofilos Papapanagiotou

Taught by

CNCF [Cloud Native Computing Foundation]

Reviews

Start your review of The State and Future of Cloud-Native Model Serving

Never Stop Learning.

Get personalized course recommendations, track subjects and courses with reminders, and more.

Someone learning on their laptop while sitting on the floor.