Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach

This seminar presents a novel language model architecture that scales test-time computation through latent space reasoning. Join speaker Jonas Geiping as he explores a recurrent block approach that can unroll to arbitrary depth at test-time, contrasting with mainstream reasoning models that scale by producing more tokens. Learn about this innovative method that doesn't require specialized training data, works with small context windows, and captures reasoning types not easily represented in words. Discover how a 3.5 billion parameter model trained on 800 billion tokens can dramatically improve performance on reasoning benchmarks, achieving computation equivalent to 50 billion parameters. The presentation covers the complete research findings detailed in the associated paper available on arXiv.