This course covers the learning outcomes and goals of unsupervised and semi-supervised learning of structure in Neural Networks for NLP. Students will learn about different methods for semi-supervised and unsupervised learning, design decisions for unsupervised models, and examples of unsupervised learning. The course teaches skills such as supervised, unsupervised, and semi-supervised learning, feature learning, and design decisions for unsupervised models. The teaching method includes lectures and examples. The intended audience for this course is students interested in neural networks for natural language processing.
Neural Nets for NLP 2020 - Unsupervised and Semi-supervised Learning of Structure
Graham Neubig via YouTube
Overview
Syllabus
Supervised, Unsupervised, Semi-supervised
Learning Features vs. Learning Discrete Structure
Unsupervised Feature Learning (Review)
How do we Use Learned Features?
What About Discrete Structure?
What is our Objective?
A Simple First Attempt
Problem: Embeddings May Not be Indicative of Syntax
Normalizing Flow (Rezende and Mohamed 2015)
Cross-lingual Application of Unsupervised Models (He et al. 2019)
Soft vs. Hard Tree Structure
One Other Paradigm: Weak Supervision
Gated Convolution (Cho et al. 2014)
Learning with RL (Yogatama et al. 2016)
Difficulties in Learning Latent Structure
Taught by
Graham Neubig
Related Courses
-
Neural Nets for NLP 2019 - Unsupervised and Semi-supervised Learning of Structure
-
Neural Nets for NLP 2017 - Unsupervised Learning of Structure
-
CMU Neural Nets for NLP 2020 - Multitask and Multilingual Learning
-
Introduction to Scientific Machine Learning
-
Neural Nets for NLP - Latent Variable Models
-
Neural Nets for NLP - Models with Latent Random Variables
