Understanding High-Dimensional Bayesian Optimization

This seminar talk by Leonard Papenmeier explores the surprising effectiveness of simple Bayesian optimization methods in high-dimensional real-world tasks, challenging previous assumptions in the field. Discover the fundamental challenges in high-dimensional Bayesian optimization (HDBO) and learn why recent methods succeed despite theoretical contradictions. Examine how two types of vanishing gradients caused by Gaussian process (GP) initialization schemes significantly impact the performance of HDBO approaches. Understand why methods promoting local search behaviors perform better in high-dimensional spaces, and see how a simple variant of maximum likelihood estimation of GP length scales achieves state-of-the-art performance across real-world applications. Consider the provocative question of whether HDBO can now be considered a solved problem. The talk is based on research published in a paper available at https://arxiv.org/abs/2502.09198.