This course aims to teach learners how to embed permutations into a continuous space and make ranking operators differentiable for integration into differentiable architectures for machine learning. The course covers topics such as permutations as inputs, SUQUAN embedding, Kendall embedding, optimal transport, differentiable sort and rank, and applications of soft quantization and soft quantiles. The teaching method involves presenting theoretical concepts and practical examples. The intended audience includes faculty, postdoctoral researchers, and Ph.D. students from the UK/EU interested in the intersection of statistics and computer science in machine learning.
Learning From Ranks, Learning to Rank - Jean-Philippe Vert, Google Brain
Alan Turing Institute via YouTube
Overview
Syllabus
Intro
Differentiable programming
Beyond images and strings
What if inputs or outputs are permutations?
Examples
Goals
Permutations as inputs
SUQUAN embedding (Le Morvan and Vert, 2017)
Supervised ON (SUQUAN)
Experiments: CIFAR-10
Limits of the SUQUAN embedding
The Kendall embedding (Jiao and Vert, 2015, 2017)
Geometry of the embedding
Kendall and Mallows kernels
Applications
Remark
Permutations as intermediate / output?
Optimal transport (OT)
Differentiable permutation matrix
Differentiable sort and rank
Soft quantization and soft quantiles
Application: soft top-k loss
Application: learning to sort
Conclusion
Taught by
Alan Turing Institute
