Foundations of Local Large Language models

Overview

Class Central Tips

By the end of this course, a learner will have a solid understanding of Large Language Models running locally. You'll be able to setup a local environment using powerful tooling to run different LLMs and interact with them both with a web interface as well as with APIs. You will explore other tools and programming languages to interact with these LLMs and using LLMs via via Hugging Face Candle and Mozilla llamafile.

Syllabus

Local LLMOps

This week, you will learn mitigation strategies, evaluate task performance, and operationalize workflows by identifying risks in notebooks and deploying an LLM application.

Production Workflows and Performance of LLMs

This week, you will explore different types of generative AI applications, including API-based, embedded model, and multi-model systems. You'll learn the fundamentals of building robust applications using techniques like Retrieval Augmented Generation (RAG) to improve context. Through hands-on exercises, you'll gain experience evaluating real-world performance of large language models using Elo ratings coded in Python, Rust, R, and Julia. Then you'll explore production LLM workflows using tools like skypilot, Lorax, and Ludwig for fine-tuning models like Mistral-7b. Finally, you'll gain hands-on experience testing an application locally and deploying it on the cloud.

Responsible Generative AI

This week you will learn foundations of generative AI and responsible deployment strategies to benefit from the latest advancements while maintaining safety, accuracy, and oversight. By directly applying concepts through hands-on labs and peer discussions, you will gain practical experience putting AI into production.