Speaker |
Michael Dennis |
|---|---|
Affiliation |
Google Deepmind |
Date |
Wednesday, 14 January 2026 |
Time |
12:30-13:30 |
Location |
Ground floor lecture theatre, Sainsbury Wellcome Center, 25 Howland St, W1T 4JG |
Link |
https://ucl.zoom.us/j/99748820264 |
Event series |
Jump Trading/ELLIS CSML Seminar Series |
Abstract |
There is immense value in using RL for resettable virtual environments to improve tool-use, computer-use, Math, and Coding. Even in physical settings, the Genie models have paved a path for training within resettable virtual environments. This presents an opportunity for setter-solver algorithms, like those used in Unsupervised Environment Design (UED) to drive efficient learning, transfer, and to lead to open-ended discovery. In this talk, we address several challenges with generalising setter-solver auto-curricula past the symmetric 2-player zero-sum setting which drove AlphaGo. We present a setter-solver algorithm (PAIRED) fit for asymmetric settings where the setter may be able to create unsolvable tasks for the solver. We then discuss a refinement of this approach (ReMiDi) which performs better in the presence of trade-offs between tasks, ensuring that progress continues until there is no way to improve the policy on any task without decreasing it on some other task. Finally, we present an approach to generalise arbitrary setter-solver algorithms to general-sum games (Rational Policy Gradient) while preventing the setter from producing opponents which self-sabotage -- which go against their incentives in the underlying game -- thus ensuring the setter only produces rational opponents. Together, by generalizing setter-solver algorithms beyond narrow games, these methods lay the groundwork for setter solver algorithms to drive discovery in general open-ended environments. |
Biography |
I'm currently a Research Scientist on Google Deepmind's Openendedness team. I was previously a Ph.D. Student at the Center for Human Compatible AI (CHAI) advised by Stuart Russell. Prior to research in AI I conducted research on computer science theory and computational geometry. |