How to Evaluate Your Evals

In this 13-minute conference talk from Conf42 LLMs 2025, Hilik Paz explores the critical process of evaluating LLM evaluation systems. Learn about the fundamentals of evals in generative AI, the unique challenges faced when testing AI systems, and practical implementation strategies. Follow along with a real-world technical support bot example that demonstrates different evaluation types and methodologies. Watch a live experiment with actual results that showcases how to properly assess AI performance. The presentation provides a structured approach to creating effective evaluation frameworks, helping developers and organizations ensure their LLM applications meet quality standards.