Explore techniques for making large language model (LLM) inference more affordable and efficient in this 32-minute conference talk by Daniel Campos at the LLMs in Production Conference. Learn about the challenges of using foundational models and APIs, and discover alternatives like self-hosting models. Delve into methods for optimizing model performance within latency and inference budgets, including pseudo-labeling, knowledge distillation, pruning, and quantization. Gain insights from Campos' extensive experience in NLP, ranging from his work at Microsoft on Bing's ranking system to his current Ph.D. research on efficient LLM inference and robust dense retrieval at the University of Illinois Urbana Champaign.
Overview
Syllabus
Making LLM Inference Affordable // Daniel Campos // LLMs in Production Conference Part 2
Taught by
MLOps.community
Related Courses
-
Efficiently Serving LLMs
-
2025 Fine Tuning LLM with Hugging Face Transformers for NLP
-
Compressing Large Language Models (LLMs) with Python Code - 3 Techniques
-
AWQ for LLM Quantization - Efficient Inference Framework for Large Language Models
-
Databricks' vLLM Optimization for Cost-Effective LLM Inference - Ray Summit 2024
-
Building LLM Applications for Production
