Overview
This course covers the fundamentals of neural networks for natural language processing (NLP) with a focus on conditioned generation. By the end of the course, students will be able to understand and implement language models, conditioned language models, ensembling techniques, parameter averaging, ensemble distillation, stacking, and basic evaluation paradigms for NLP tasks. The course teaches skills such as ancestral sampling, linear and log-linear modeling, and various evaluation methods including human evaluation and perplexity. The teaching method involves theoretical explanations, practical examples, and discussions on evaluating unconditioned generation. This course is intended for individuals interested in NLP, neural networks, and machine learning.
Syllabus
Intro
Language Models Language models are generative models of text
Conditioned Language Models
Ancestral Sampling
Ensembling
Linear or Log Linear?
Parameter Averaging
Ensemble Distillation (e.g. Kim et al. 2016)
Stacking
Basic Evaluation Paradigm
Human Evaluation
Perplexity
A Contrastive Note: Evaluating Unconditioned Generation
Taught by
Graham Neubig