Paper · Oct 30, 2023

Metric Flows with Neural Networks

James Halverson, Fabian Ruehle

hep-tharXiv:2310.19870v2
View PDF View HTML arXiv page

As on arXiv

Abstract

We develop a general theory of flows in the space of Riemannian metrics induced by neural network gradient descent. This is motivated in part by recent advances in approximating Calabi-Yau metrics with neural networks and is enabled by recent advances in understanding flows in the space of neural networks. We derive the corresponding metric flow equations, which are governed by a metric neural tangent kernel, a complicated, non-local object that evolves in time. However, many architectures admit an infinite-width limit in which the kernel becomes fixed and the dynamics simplify. Additional assumptions can induce locality in the flow, which allows for the realization of Perelman's formulation of Ricci flow that was used to resolve the 3d Poincaré conjecture. We demonstrate that such fixed kernel regimes lead to poor learning of numerical Calabi-Yau metrics, as is expected since the associated neural networks do not learn features. Conversely, we demonstrate that well-learned numerical metrics at finite-width exhibit an evolving metric-NTK, associated with feature learning. Our theory of neural network metric flows therefore explains why neural networks are better at learning Calabi-Yau metrics than fixed kernel methods, such as the Ricci flow.

View PDF View HTML arXiv page
Submitted
Oct 30, 2023
Last revised
Oct 18, 2024 (v2)
Subjects
hep-th, cs.LG, math.DG
DOI
10.1088/2632-2153/ad8533

Comments: 32 pages + references. Published in Machine Learning: Science and Technology

Ricci flowRicci-flat & Calabi–YauPoincaré conjecture3-manifoldsKähler geometryDiscrete & graph RicciNumerics & visualization

Its neighbourhood

Connected papers

The 8 papers on this site closest to this one, judged by the topics they share, the words and phrases in their titles and abstracts, and their authors. They are not its citations, but a map of the work around it. Papers here go back to 2021, plus the canon.

Connected papers This paper in the centre and the 8 papers closest to it around it, in date order clockwise from the top. The closer a paper sits to the centre, the stronger the connection. This paper Ricci Flow and the Poincare Conjecture (John W. Morgan and Gang Tian, 2006)6Morgan 2006 Dynamically Stable Poincaré Embeddings for Neural Manifolds (Jun Chen, Yuang Liu, Xiangrui Zhao et al., 2021)3Chen 2021 On 4-dimensional smooth Poincare conjecture (Selman Akbulut, 2022)8Akbulut 2022 A physics-informed search for metric solutions to Ricci flow, their embeddings, and visualisation (Aarjav Jain, Challenger Mishra and Pietro Liò, 2022)2Jain 2022 Rigor with Machine Learning from Field Theory to the Poincaré Conjecture (Sergei Gukov, James Halverson and Fabian Ruehle, 2024)1Gukov 2024 Deep Learning as Ricci Flow (Anthony Baptista, Alessandro Barp, Tapabrata Chakraborti et al., 2024)5Baptista 2024 Ricci Flow on Weighted Digraphs with Balancing Factor (Shuliang Bai, Rui Li, Shuang Liu et al., 2025)7Bai 2025 Neural Feature Geometry Evolves as Discrete Ricci Flow (Moritz Hehl, Max von Renesse and Melanie Weber, 2025)4Hehl 2025
Closer to the centre: more closely connected. Clockwise from the top: oldest to newest. Ricci flowWider flowsalso connected to each other
  1. Rigor with Machine Learning from Field Theory to the Poincaré Conjecture

    Sergei Gukov, James Halverson and Fabian Ruehle

    Topics Poincaré conjecture, Discrete & graph Ricci · Same authors James Halverson, Fabian Ruehle · Shared terms “neural networks”, “learning”, “gradient descent”

  2. A physics-informed search for metric solutions to Ricci flow, their embeddings, and visualisation

    Aarjav Jain, Challenger Mishra and Pietro Liò

    Topics Numerics & visualization, Discrete & graph Ricci, Ricci-flat & Calabi–Yau · Shared terms “neural networks”, “Yau metrics”, “numerical”

  3. Dynamically Stable Poincaré Embeddings for Neural Manifolds

    Jun Chen, Yuang Liu, Xiangrui Zhao et al.

    Topics Discrete & graph Ricci · Shared terms “neural networks”, “learning”, “Poincaré”

  4. Neural Feature Geometry Evolves as Discrete Ricci Flow

    Moritz Hehl, Max von Renesse and Melanie Weber

    Topics Discrete & graph Ricci · Shared terms “neural networks”, “learn feature”, “understanding”

  5. Deep Learning as Ricci Flow

    Anthony Baptista, Alessandro Barp, Tapabrata Chakraborti et al.

    Topics Discrete & graph Ricci · Shared terms “neural networks”, “learning”, “understanding”

  6. Ricci Flow and the Poincare Conjecture

    John W. Morgan and Gang TianThe canon

    Topics Poincaré conjecture · Shared terms “Perelman”

  7. Ricci Flow on Weighted Digraphs with Balancing Factor

    Shuliang Bai, Rui Li, Shuang Liu et al.

    Topics Numerics & visualization, Discrete & graph Ricci · Shared terms “learning”, “numerical”, “demonstrate”

  8. On 4-dimensional smooth Poincare conjecture

    Selman Akbulut

    Topics Poincaré conjecture

Read around it

Background reading

The guides for the topics of this paper: what each idea is, why it matters, and the papers to read first.

Ricci flow Hamilton's Ricci flow evolves a Riemannian metric by minus twice its Ricci curvature. It is a nonlinear heat equation for geometry: it spreads… Start with: Three-manifolds with positive Ricci curvature (Hamilton, 1982) Ricci-flat & Calabi–Yau Ricci-flat metrics do not move under Ricci flow. Every known compact simply connected example has special holonomy (Calabi–Yau, hyperkähler, G₂… Start with: On the ricci curvature of a compact kähler manifold and the complex monge‐ampére equation, I (Yau, 1978) Poincaré conjecture Every closed, simply connected 3-manifold is homeomorphic to the 3-sphere. Poincaré posed the question in 1904, and Perelman proved it in 2002–03… Start with: Generalized Poincare's Conjecture in Dimensions Greater Than Four (Smale, 1961) 3-manifolds Dimension three is where Ricci flow has done its most complete work. It proves geometrization, describes the flow's long-time behaviour, and now… Start with: Three-manifolds with positive Ricci curvature (Hamilton, 1982)

In the guidebook

  1. 10A.4
    Homotopy Groups, Degree and the Hurewicz TheoremBook 10A · Three-Manifolds and Geometrization

    Spheres mapped into manifolds: the algebraic topology that finite extinction runs on.

  2. 10A.6
    Necks, Caps and Their TopologyBook 10A · Three-Manifolds and Geometrization

    What an ε-neck is, how necks overlap and chain into tubes, and which manifolds necks and caps can build.

  3. 12C.1
    Reading Off the TopologyBook 12C · Extinction and the Proof

    What survives the flow, and why only S³ remains.