\dm_csml_event_details UCL ELLIS

Efficient score estimation with infinite-dimensional exponential families


Speaker

Dougal Sutherland

Affiliation

UCL, Gatsby

Date

Friday, 15 December 2017

Time

13:00-14:00

Location

Zoom

Link

Roberts Building G08 Sir David Davies LT

Event series

DeepMind/ELLIS CSML Seminar Series

Abstract

The infinite-dimensional exponential family is a rich generalization of the standard exponential family, going beyond finite sufficient statistic functions to allow for very complex models. In particular, we study the kernel exponential family, where the natural parameter lies in a reproducing kernel Hilbert space. Computing the normalization constant in this class of models is difficult, but efficient estimation is possible via score matching. This approach, however, has cubic computational complexity in both the number of sampled points and their dimension. We thus propose estimation with a low-rank, Nyström-like approximation. The new solution retains essentially the same convergence rate of the full-rank solution, with substantially less computational effort and storage. We demonstrate the applicability of the method both to density estimation and to approximating Hamiltonian Monte Carlo when gradients are available.

Biography