Class Central is learner-supported. When you buy through links on our site, we may earn an affiliate commission.

IBM

Mastering Generative AI: Advanced Fine-Tuning for LLMs

IBM via edX

Overview

Employers are actively hunting for AI engineers who know how to fine-tune transformers for gen AI applications. This Mastering Generative AI - Advanced Fine-Tuning for LLMs course is designed to give AI engineers and other AI specialists the highly sought-after skills employers need.

AI engineers use advanced fine-tuning skills for LLMs to tailor pre-trained models for specific tasks to ensure accuracy and relevance in applications like chatbots, translation, and content generation.

During this course, you’ll explore the basics of instruction-tuning with Hugging Face, reward modeling, and training a reward model. You’ll look at proximal policy optimization (PPO) with Hugging Face and its configuration, large language models (LLMs) as distributions, and reinforcement learning from human feedback (RLHF). Plus, you’ll investigate direct performance optimization (DPO) with Hugging Face using the partition function.

As you progress through the course, you’ll also build your practical hands-on experience in online labs where you’ll work on reward modeling, PPO, and DPO.

If you’re keen to extend your gen AI engineering skills to include advanced fine-tuning for LLMs so you can catch the eye of an employer, ENROLL TODAY and power up your resume in just 2 weeks!

Prerequisites: To take this course, you need knowledge of LLMs, instruction-tuning, and reinforcement learning. Familiarity with machine learning and neural network concepts is useful too.

Syllabus

Module 0: Welcome

  • Video: Course Introduction
  • Reading: Professional Certificate Overview
  • Reading: General Information
  • Reading: Learning Objectives and Syllabus
  • Reading: Grading Scheme

Module 1: Different Approaches to Fine-Tuning

  • Reading: Module Introduction and Learning Objectives
  • Video: Basics of Instruction-Tuning
  • Video: Instruction-Tuning with Hugging Face
  • Lab: Instruction Fine-Tuning LLMs
  • Video: Reward Modeling: Response Evaluation
  • Video: Reward Model Training
  • Video: Reward Modeling with Hugging Face
  • Lab: Reward Modeling
  • Practice Quiz: Instruction-Tuning and Reward Modeling
  • Reading: Summary and Highlights
  • Graded Quiz: Different Approaches to Instruction-Tuning

Module 2: Fine-Tuning Causal LLMs with Human Feedback and Direct Preference

  • Reading: Module Introduction and Learning Objectives
  • Video: Large Language Models (LLMs) as Distributions
  • Video: From Distributions to Policies
  • Video: Reinforcement Learning from Human Feedback (RLHF)
  • Video: Proximal Policy Optimization (PPO)
  • Video: PPO with Hugging Face
  • Video: PPO Trainer
  • Lab: Reinforcement Learning from Human Feedback using PPO
  • Reading: Summary and Highlights
  • Practice Quiz: Proximal Policy Optimization (PPO)
  • Video: DPO: Partition Function
  • Video: DPO: Optimal Solution
  • Video: DPO with Hugging Face
  • Lab: Direct Preference Optimization (DPO) using Hugging Face
  • Lab: Fine-tune LLMs Locally with InstructLab
  • Reading: Summary and Highlights
  • Practice Quiz: Direct Preference Optimization (DPO)
  • Graded Assessment: Fine-Tuning Causal LLMs with Human Feedback and Direct Preference
  • Reading: Cheat Sheet: Generative AI Advanced Fine-Tuning for LLMs
  • Reading: Course Glossary: Generative AI Advance Fine-Tuning for LLMs

Course Wrap-Up

  • Course Conclusion
  • Reading: Congratulations and Next Steps
  • Reading: Team and Acknowledgements
  • Reading: Copyrights and Trademarks
  • Course Rating and Feedback
  • Reading: Frequently Asked Questions
  • Reading: Claim your badge here

Taught by

Joseph Santarcangelo

Reviews

Start your review of Mastering Generative AI: Advanced Fine-Tuning for LLMs

Never Stop Learning.

Get personalized course recommendations, track subjects and courses with reminders, and more.

Someone learning on their laptop while sitting on the floor.