As on arXiv
Abstract
Deep neural networks (DNNs) are powerful tools for approximating the distribution of complex data. It is known that data passing through a trained DNN classifier undergoes a series of geometric and topological simplifications. While some progress has been made toward understanding these transformations in neural networks with smooth activation functions, an understanding in the more general setting of non-smooth activation functions, such as the rectified linear unit (ReLU), which tend to perform better, is required. Here we propose that the geometric transformations performed by DNNs during classification tasks have parallels to those expected under Hamilton's Ricci flow - a tool from differential geometry that evolves a manifold by smoothing its curvature, in order to identify its topology. To illustrate this idea, we present a computational framework to quantify the geometric changes that occur as data passes through successive layers of a DNN, and use this framework to motivate a notion of `global Ricci network flow' that can be used to assess a DNN's ability to disentangle complex data geometries to solve classification problems. By training more than DNN classifiers of different widths and depths on synthetic and real-world data, we show that the strength of global Ricci network flow-like behaviour correlates with accuracy for well-trained DNNs, independently of depth, width and data set. Our findings motivate the use of tools from differential and discrete geometry to the problem of explainability in deep learning.
Its neighbourhood
Connected papers
The 8 papers on this site closest to this one, judged by the topics they share, the words and phrases in their titles and abstracts, and their authors. They are not its citations, but a map of the work around it. Papers here go back to 2021, plus the canon.
-
Neural Feature Geometry Evolves as Discrete Ricci Flow
Topics Discrete & graph Ricci, Weak & singular flows · Shared terms “neural networks”, “geometric transformations”, “activation”
-
Dynamically Stable Poincaré Embeddings for Neural Manifolds
Topics Discrete & graph Ricci · Shared terms “neural networks”, “learning”, “classification tasks”
-
Geometric Meta-Learning via Coupled Ricci Flow: Unifying Knowledge Representation and Quantum Entanglement
Topics Discrete & graph Ricci · Shared terms “deep learning”, “neural networks”, “topological simplification”
-
Extensions and Applications of Bredon's Trick in Geometric and Topological Contexts
Topics Discrete & graph Ricci · Shared terms “neural networks”, “computational”, “differential geometry”
-
Metric Flows with Neural Networks
Topics Discrete & graph Ricci · Shared terms “neural networks”, “learning”, “understanding”
-
Stability of multiphase mean curvature flow beyond circular topology changes
Topics Phase fields & multiphase flows, Weak & singular flows · Shared terms “illustrate”
-
Convergence of a vector-valued Allen-Cahn system to Brakke's multiphase mean curvature flow
Topics Phase fields & multiphase flows, Weak & singular flows
-
Type-0 singularities in the network flow – Evolution of trees
Topics Phase fields & multiphase flows · Shared terms “network flow”
Read around it
Background reading
The guides for the topics of this paper: what each idea is, why it matters, and the papers to read first.
In the guidebook
- 6D.11Computing Geometric Flows
Numerical curve shortening and mean curvature flow, level sets and phase fields, discrete Ricci flow on surfaces and in conformal mapping, rotationally symmetric neckpinches, and what computation has taught about singularities.
- 12C.9After Perelman
Singular flows, the Smale conjecture, and Bamler’s theory.
- 5A.5Uniformization and the Two-Dimensional Ricci Flow
Constant-curvature metrics on surfaces, and Ricci flow as a heat equation for the conformal factor.