Deep learning and Bayesian machine learning are currently two of the most
active areas of machine learning research. Deep learning provides a
powerful class of models and an easy framework for learning that now
provides state-of-the-art methods for applications ranging from image
classification to speech recognition. Bayesian reasoning provides a
powerful approach for knowledge integration, inference, and decision making
that has established it as the key tool for data-efficient learning,
uncertainty quantification and robust model composition, widely-used in
applications ranging from information retrieval to large-scale ranking.
Each of these research areas has shortcomings that can be effectively
addressed by the other, pointing towards a needed convergence of these two
areas of machine learning, and that enhances our machine learning practice. One powerful outcome of this convergence is our ability to develop systems
for probabilistic inference with memory. A memory-based inference amortises
the cost of probabilistic reasoning by cleverly reusing prior computations.
To explore this, we shall take a statistical tour of deep learning,
re-examine latent variable models and approximate Bayesian inference, and
make connections to denoising auto-encoders and other stochastic
encoder-decoder systems. In this way, we will make sense of what memory in
inference might mean, and highlight the use of amortised inference in many
other parts of machine learning. ---- Bio ----
Shakir's research focuses on exploring and incorporating probabilistic
reasoning in all aspects of machine learning, towards the goal of building
principled, scalable and general-purpose probabilistic decision-making
systems. His current research interests lie at the intersection of
variational inference, deep learning and reinforcement learning. Shakir is
a senior research scientist at Google DeepMind in London. Before moving to
London, he held a junior research fellowship from the Canadian Institute
for Advanced Research (CIFAR) as part of the programme on Neural
Computation and Adaptive Perception, at the University of British Columbia
with Nando de Freitas. He completed his PhD with Zoubin Ghahramani at the
University of Cambridge, as a Commonwealth Scholar to the United Kingdom
and a member of St John's College. He is from South Africa, and completed
his prior degrees in Electrical and Information Engineering at the
University of the Witwatersrand, Johannesburg. |