Overview
Master Apple MLX fine-tuning in this comprehensive 47-minute tutorial that walks through essential techniques for training custom models. Begin with MLX installation and basic prompting before diving into working with Qwen 2.5 Coder models. Learn crucial skills for dataset construction, including achieving proper balance and removing unintended consequences. Explore the differences between full fine-tuning and LoRA approaches while comparing various model sizes from 500M to 7B parameters. Discover strategies for maintaining model knowledge during training through mixer techniques, and understand how to optimize models for chat applications. Get hands-on experience with dataset diversity management, model fusing, and practical implementation using the provided GitHub repository. Perfect for developers looking to enhance their machine learning capabilities on Apple's MLX framework while avoiding common pitfalls in model training and optimization.
Syllabus
- intro
- installing mlx
- prompting with mlx
- Qwen 2.5 Coder
- building a dataset
- fine tuning a qwen-2.5-coder-7b model
- comparing a 500M parameter model
- fine tuning with lora
- fixing dataset diversity
- fixing model forgetfulness using mixers
- Qwen 2.5 Code 3B
- Fine tuning for chat
- Fusing Models
Taught by
Chris Hay