Class Central is learner-supported. When you buy through links on our site, we may earn an affiliate commission.

CNCF [Cloud Native Computing Foundation]

Harnessing the Power of Envoy Proxy for Building an LLM Gateway

CNCF [Cloud Native Computing Foundation] via YouTube

Overview

Learn how to leverage Envoy Proxy's unique architecture and capabilities to build a robust LLM gateway in this 38-minute conference talk. Discover why Envoy Proxy surpasses traditional proxies like NGINX through its extensibility features, including filter architecture, callout architecture (ext-proc, ext-auth), and dynamic library loading capabilities. Explore practical applications of Envoy proxy for managing LLM credentials, implementing prompt guarding/decorating, analyzing content safety, controlling usage, enabling context-aware failover, and enhancing observability. Gain insights into how Envoy's high-performance, asynchronous C++ core enables it to function effectively as an ingress, egress, and mesh gateway. Particularly valuable for developers, architects, and technology professionals seeking solutions for LLM implementation challenges and infrastructure platform optimization.

Syllabus

Harnessing the Power of Envoy Proxy for Building an LLM Gateway - Idit Levine, Solo.io

Taught by

CNCF [Cloud Native Computing Foundation]

Reviews

Start your review of Harnessing the Power of Envoy Proxy for Building an LLM Gateway

Never Stop Learning.

Get personalized course recommendations, track subjects and courses with reminders, and more.

Someone learning on their laptop while sitting on the floor.