Speaker |
Long Tran-Thanh |
|---|---|
Affiliation |
University of Warwick |
Date |
Wednesday, 13 May 2026 |
Time |
12:30-13:30 |
Location |
Ground floor lecture theatre, Sainsbury Wellcome Center, 25 Howland St, W1T 4JG |
Link |
https://ucl.zoom.us/j/99748820264 |
Event series |
Jump Trading/ELLIS CSML Seminar Series |
Abstract |
Sparse neural networks promise inference-time efficiency, yet training them effectively remains a fundamental challenge. Despite advances in pruning methods that create sparse architectures, understanding why some sparse structures are better trainable than others with the same level of sparsity remains poorly understood. Aiming to develop a systematic approach to this fundamental problem, we propose a novel theoretical framework based on the theory of graph limits, particularly graphons, that characterises sparse neural networks in the infinite-width regime. Our key insight is that connectivity patterns of sparse neural networks induced by pruning methods converge to specific graphons as networks' width tends to infinity, which encodes implicit structural biases of different pruning methods. Based on this, we derive a Graphon Neural Tangent Kernel (Graphon NTK) to study the training dynamics of sparse networks in the infinite width limit. Graphon NTK provides a general framework for the theoretical analysis of sparse networks. We empirically show that the spectral analysis of Graphon NTK correlates with observed training dynamics of sparse networks, explaining the varying convergence behaviours of different pruning methods. In addition, we also prove two fundamental theoretical results: (i) a Universal Approximation Theorem for sparse networks that depends only on the intrinsic dimension of active coordinate subspaces; and (ii) a Graphon-NTK generalisation bound demonstrating how the limit graphon modulates the kernel geometry to align with informative features. Overall, our framework provides theoretical insights into the impact of connectivity patterns on the trainability of various sparse network architectures. As such, it transforms the study of sparse neural networks from combinatorial graph problems into a rigorous framework of continuous operators, offering a new mechanism for analysing expressivity and generalisation in sparse neural networks. |
Biography |
Long is a Full Professor at the Department of Computer Science, University of Warwick, UK. He is currently the Director of Research of the department (Deputy-Head) and the university’s Chair of Digital Research Spotlight. Long has been doing active research in a number of key areas of Artificial Intelligence and multi-agent systems, mainly focusing on multi-armed bandits, game theory, and incentive engineering, and their applications to AI for Social Good. He has published more than 90 papers at peer-reviewed A* conferences in AI/ML (including AAAI, AAMAS, CVPR, ICLR, IJCAI, NeurIPS) and journals (JAAMAS, AIJ), and have received a number of prestigious national/international awards, including 2 best paper honourable mention awards at top-tier AI conferences (AAAI, ECAI), 2 Best PhD Thesis Award Honourable Mentions (UK's BCS and Europe’s ECCAI/EurAI), and the co-recipient of the 2021 AIJ Prominent Paper Award (for one of the 2 most influential papers between 2014-2021 published at the Artificial Intelligence Journal). Long has also been actively involved in a number of community services, including being the local co-chair for AAMAS 2021, AAMAS 2023, KR 2021, KR 2024, and AAMAS 2027. He is an Associate Editor for JAAMAS, the flagship multi-agent systems journal, and an Associate Editor for AIJ, the flagship AI journal. Previously he was a member of the IFAAMAS Board of Directors between 2018-2024 and was a Turing Fellow at the Alan Turing Institute, UK. |