Fine-tuning LLMs on Human Feedback (RLHF + DPO)

This tutorial explores the techniques for fine-tuning Large Language Models (LLMs) using human feedback, covering both Reinforcement Learning from Human Feedback (RLHF) and Direct Policy Optimization (DPO). Learn about the evolution from base models to InstructGPT, understand the mechanics of RLHF and its limitations, and discover how DPO offers a more efficient alternative. Follow along with a practical example of fine-tuning the Qwen model on title preferences through a three-step process: curating preference data, implementing DPO fine-tuning, and evaluating the resulting model. The video includes timestamps for easy navigation through topics like Proximal Policy Optimization (PPO) and provides links to relevant research papers, datasets, and the fine-tuned model on Hugging Face.