Calculus and Fixed Points in Banach Spaces

4A.1 Banach Spaces and Bounded Operators and 4A.2 Baire Category and Its Consequences studied linear maps between Banach spaces. Most equations of geometry and physics are nonlinear: $-\Delta u + u^3 = f$, $\partial_tg = -2\operatorname{Ric}(g)$, the equations of an elastic rod under load. The basic strategy for nonlinear problems is the one of first-year calculus: linearise. Near a known solution, replace the nonlinear map by its derivative, solve the linear problem, and argue that the true solution is close. In finite dimensions that strategy is the inverse function theorem. This chapter proves that theorem in Banach spaces, where the "variables" are functions and the derivative is a linear operator, and shows what it does: it solves nonlinear boundary value problems near known solutions, tracks eigenvalues as an operator is perturbed, explains why solutions to an equation can suddenly branch (an elastic column buckling), and underlies every short-time existence theorem in this guide, including the one for the Ricci flow (11A.4 Short-Time Existence and Uniqueness).

Everything rests on one theorem with a one-paragraph proof: the contraction mapping principle. A map that shrinks distances by a fixed factor has exactly one fixed point, found by iterating. Completeness, the defining property of a Banach space (4A.1 Banach Spaces and Bounded Operators), is exactly what makes the iteration converge.

By the end of this chapter you will be able to:

• define and compute Fréchet derivatives, distinguish them from directional (Gâteaux) derivatives, and prove the chain rule and the mean value inequality;
• prove the contraction mapping principle, with continuous dependence on parameters, and use it to solve integral equations;
• prove the inverse and implicit function theorems in Banach spaces, with explicit radii, and analyse Newton's method;
• prove the Picard–Lindelöf theorem for ODEs in Banach spaces;
• apply the implicit function theorem to nonlinear boundary value problems and to the perturbation of simple eigenvalues;
• explain bifurcation from a simple eigenvalue through Lyapunov–Schmidt reduction, with Euler's buckling column as the example, and explain why some geometric problems "lose derivatives" and need the Nash–Moser theorem.

3.1Solving by linearising

3.2Derivatives in Banach spaces

Let $X$, $Y$ be Banach spaces, $U \subseteq X$ open, and $F : U \to Y$.

The definition is the one of 2B.8 Calculus in Several Variables word for word, with norms in place of absolute values. Uniqueness: if $A$ and $A'$ both work, then $\|(A - A')h\| = o(\|h\|)$, and replacing $h$ by $th$ with $t \to 0$ shows $(A - A')h = 0$.

A weaker notion is the Gâteaux (directional) derivative: $\lim_{t\to0}\frac1t(F(x + th) - F(x))$ exists for each $h$ and is a bounded linear function of $h$. Fréchet differentiability implies Gâteaux differentiability with the same derivative; the converse fails, already in two dimensions (2B.8 Calculus in Several Variables), and in infinite dimensions in more interesting ways (Exercise 3.12).

Example 3 is a warning in disguise. The same formula on $L^2(0, 1)$ fails: a Nemytskii operator $u \mapsto \phi(u)$ maps $L^2$ to itself and is Fréchet differentiable there only if $\phi$ is affine (a theorem of Krasnosel'skii; Exercise 3.12 shows the failure for $\sin$). The sup norm controls $h$ everywhere, which Taylor's theorem needs; the $L^2$ norm allows $h$ to be large on a small set. This is one reason why nonlinear PDE are often set in Hölder spaces (4A.14 Hölder Spaces) or in Sobolev spaces above the embedding threshold (4A.11 Sobolev Embeddings and Critical Exponents), where functions are continuous.

There is no mean value theorem in Banach spaces (already for maps $\mathbb{R} \to \mathbb{R}^2$, $t \mapsto (\cos t, \sin t)$ on $[0, 2\pi]$ has $F(2\pi) - F(0) = 0$ while $F'$ never vanishes). What survives, and suffices, is an inequality.

(Hahn–Banach is a forward reference; without it, integrate: $F(y) - F(x) = \int_0^1DF(x + s(y - x))(y - x)\,ds$, a Riemann integral of a continuous Banach-space-valued function, and estimate the norm of the integral by the integral of the norm.)

Higher derivatives are defined by iteration: $D^2F(x) \in \mathcal L(X, \mathcal L(X, Y))$, identified with bounded bilinear maps $X\times X \to Y$, symmetric when $F$ is $C^2$. Taylor's theorem with remainder holds, with the same proof via the integral form.

3.3The contraction mapping principle

The error bound is computable before any iteration, which is why the principle is the basis of validated numerics as well as of existence theory. Two refinements are used constantly.

• Closed subsets. If $T$ maps a closed subset $K$ of a Banach space into itself and is a contraction there, the theorem applies on $K$, which is complete. In practice $K$ is a closed ball, and one checks separately that $T$ maps the ball into itself and contracts on it.
• Parameters. If $T_\lambda$ is a contraction with the same $\theta$ for each $\lambda$ in a metric space $\Lambda$, and $\lambda \mapsto T_\lambda x$ is continuous for each $x$, then the fixed point $x^*(\lambda)$ depends continuously on $\lambda$: $d(x^*(\lambda), x^*(\mu)) = d(T_\lambda x^*(\lambda), T_\mu x^*(\mu)) \leq \theta d(x^*(\lambda), x^*(\mu)) + d(T_\lambda x^*(\mu), T_\mu x^*(\mu))$, so $d(x^*(\lambda), x^*(\mu)) \leq \frac{1}{1 - \theta}d(T_\lambda x^*(\mu), T_\mu x^*(\mu)) \to 0$ as $\lambda \to \mu$.

3.4The inverse function theorem

The proof gives an explicit radius: the theorem holds on the ball where $\|DF(x) - DF(x_0)\| \leq \frac{1}{2\|DF(x_0)^{-1}\|}$. Quantitative inverse function theorems of this kind are what make perturbation arguments uniform, as in the proof that canonical neighbourhoods are open conditions (12B.4 Compactness and Universal Estimates for κ-Solutions).

Newton's method. To solve $F(x) = 0$ near a point where $DF$ is invertible, iterate $x_{k + 1} = x_k - DF(x_k)^{-1}F(x_k)$. If $F$ is $C^2$ (or $DF$ is Lipschitz with constant $L$), $x^*$ is a zero with $\|DF(x^*)^{-1}\| \leq \beta$, and $x_0$ is close enough, then

(Exercise 3.15): quadratic convergence. Kantorovich's theorem (1948) gives a version that needs no knowledge of $x^*$: if $\|DF(x_0)^{-1}F(x_0)\| \leq \eta$, $\|DF(x_0)^{-1}\| \leq \beta$, $DF$ is $L$-Lipschitz, and $\beta L\eta \leq \frac12$, then a zero exists near $x_0$ and Newton's method converges to it. Compared with the contraction iteration used in the proof above (the "simplified Newton method", which keeps $DF(x_0)$ fixed and converges only linearly), Newton's method pays for a new linear solve at each step with a much faster rate. This trade-off reappears at the end of the chapter, where it is the key to the Nash–Moser theorem.

3.5Ordinary differential equations in Banach spaces

For a locally Lipschitz $F$ the same argument gives a solution on a short interval, which continues as long as it stays bounded. For a bounded operator $A$, $\dot x = Ax$ is solved by $x(t) = e^{tA}x_0$ with $e^{tA} = \sum\frac{t^kA^k}{k!}$, convergent in operator norm.

The theorem is exactly what fails for partial differential equations written as ODEs in function spaces. The heat equation $\dot u = \Delta u$ is an "ODE" in $L^2$, but $\Delta$ is unbounded, not Lipschitz, and the series for $e^{t\Delta}$ diverges. Two ways out lead to large parts of this guide. One treats $\Delta$ as an unbounded self-adjoint operator and defines $e^{t\Delta}$ by the spectral theorem (4A.13 Unbounded Operators and the Spectral Theorem) or by semigroup theory (6B.13 Semigroups and Evolution Equations); the other writes the equation as a fixed point problem for the Duhamel formula $u(t) = e^{t\Delta}u_0 + \int_0^te^{(t - s)\Delta}N(u(s))\,ds$ and applies the contraction principle to that, as in the semilinear theory of 6A.8 Nonlinear Parabolic Equations and the short-time existence for Ricci–DeTurck flow (11A.4 Short-Time Existence and Uniqueness).

3.6Applications

A nonlinear boundary value problem. Consider $-u'' + u^3 = f$ on $(0, 1)$ with $u(0) = u(1) = 0$. Let $X = \{u \in C^2([0, 1]) : u(0) = u(1) = 0\}$, $Y = C([0, 1])$, and $F(u) = -u'' + u^3$, a $C^1$ map $X \to Y$ by Example 3.2 with $DF(0) = -\frac{d^2}{dx^2}$. That operator is an isomorphism $X \to Y$: it is bounded, and its inverse is integration against the Green's function $G(s, r) = \min(s, r)(1 - \max(s, r))$, which maps $C$ into $X$ boundedly (Exercise 3.16). By the inverse function theorem, for every $f$ with $\|f\|_\infty$ small there is a unique small solution $u \in C^2$, depending smoothly on $f$. (Here the monotone structure of $u^3$ actually gives existence for all $f$ by the direct method, 4A.7 Weak Convergence and the Direct Method; the inverse function theorem's virtue is that it needs no such structure, only an invertible linearisation.)

Perturbation of a simple eigenvalue. Let $A$ be a bounded self-adjoint operator on a Hilbert space $H$ with a simple eigenvalue $\lambda_0$, unit eigenvector $x_0$, and let $B$ be bounded self-adjoint. For which $(x, \lambda)$ near $(x_0, \lambda_0)$ is $(A + \varepsilon B)x = \lambda x$? Set

At $(0, x_0, \lambda_0)$ the derivative in $(x, \lambda)$ is $(h, \mu) \mapsto ((A - \lambda_0)h - \mu x_0, \langle h, x_0\rangle)$, which is an isomorphism $H\times\mathbb{R} \to H\times\mathbb{R}$ provided $A - \lambda_0$ is invertible on $x_0^\perp$ (true, for instance, when $\lambda_0$ is an isolated simple eigenvalue; Exercise 3.17). So there are smooth branches $\lambda(\varepsilon)$, $x(\varepsilon)$, and differentiating $(A + \varepsilon B)x(\varepsilon) = \lambda(\varepsilon)x(\varepsilon)$ at $\varepsilon = 0$ and pairing with $x_0$ gives

the Hellmann–Feynman formula of quantum chemistry and first-order perturbation theory. The same computation, with $A$ the Laplacian and $B$ a change of metric, gives the derivative of an eigenvalue of a manifold under deformation of the metric, and it explains why Perelman's $\lambda$, the bottom eigenvalue of $-4\Delta + R$, varies smoothly along the Ricci flow: it is simple, because its eigenfunction is positive (12A.2 Ricci Flow as a Gradient Flow).

3.7Bifurcation: when the derivative is not invertible

The implicit function theorem says that solutions of $F(\lambda, u) = 0$ form a single smooth branch wherever $D_uF$ is invertible. Bifurcation happens where it is not.

Lyapunov–Schmidt reduction. Suppose $F : \mathbb{R}\times X \to Y$ is smooth, $F(\lambda, 0) = 0$ for all $\lambda$, and at $\lambda_0$ the operator $L = D_uF(\lambda_0, 0)$ is Fredholm of index zero with a one-dimensional kernel spanned by $v$: its range $R$ is closed of codimension one (4A.8 Compact Operators and Spectra proves this for the identity minus a compact operator). Choose a complement and projections: $X = \operatorname{span}(v)\oplus X_1$, $Y = R\oplus\operatorname{span}(w)$, with $Q$ the projection onto $R$. Write $u = sv + x_1$. The equation $F = 0$ splits into

The first has derivative in $x_1$ equal to $QL|_{X_1} : X_1 \to R$, an isomorphism, so by the implicit function theorem it determines $x_1 = x_1(\lambda, s)$. Substituting into the second leaves one scalar equation in two scalar unknowns, $\beta(\lambda, s) = 0$: an infinite-dimensional problem reduced to a calculus problem. Since $u = 0$ always solves, $\beta(\lambda, 0) = 0$, and $\beta(\lambda, s) = s\gamma(\lambda, s)$.

Transversality says $\partial_\lambda\gamma(\lambda_0, 0) \neq 0$, so the implicit function theorem applied to $\gamma(\lambda, s) = 0$ gives $\lambda = \lambda(s)$ (Exercise 3.18). For the elastica, $v = \cos(\pi s/L)$, transversality holds, and expanding $\sin\theta$ to third order shows $\lambda(s) = P_1(1 + \frac{s^2}{8} + \dots)$ in suitable units: the branch bends to the right (supercritical pitchfork), so a slightly overloaded column bends a little and stably. Krasnosel'skii (1950s) and Rabinowitz (1971) proved global versions: bifurcation from an eigenvalue of odd multiplicity produces a continuum of solutions that is unbounded or returns to the trivial line.

Bifurcation theory appears throughout this guide. Turing patterns in reaction–diffusion systems are bifurcations from a homogeneous state (6A.9 Reaction–Diffusion: Fronts, Patterns and Pinching); the mountain pass theorem finds solutions that perturbation cannot (6C.2 Critical Points and the Mountain Pass); and in geometric analysis, branches of constant mean curvature surfaces, Einstein metrics and solitons are found or ruled out by exactly this reduction, with the linearised operator Fredholm by elliptic theory.

3.8Loss of derivatives and the Nash–Moser theorem

The inverse function theorem needs $DF(x)$ to be an isomorphism between the same spaces $X$ and $Y$ in which $F$ is $C^1$, for all $x$ near $x_0$. In some geometric problems the linearisation is invertible only as a map that loses derivatives: $DF(x)^{-1}$ maps $C^{k + \alpha}$ to $C^{k - d + \alpha}$ for some $d > 0$, not back into the space one started from. Then the contraction iteration cannot close: each step costs $d$ derivatives.

Two famous examples. John Nash's 1956 theorem on isometric embeddings of Riemannian manifolds in Euclidean space faces a linearised system whose solutions lose derivatives; Nash overcame it by alternating Newton steps with smoothing operators, which restore derivatives at the cost of a controlled error. Moser abstracted the method (1966), and Hamilton gave the general Nash–Moser inverse function theorem for "tame" maps between Fréchet spaces (1982). And the Ricci flow itself: the equation $\partial_tg = -2\operatorname{Ric}(g)$ is only weakly parabolic, its linearisation has a kernel coming from diffeomorphisms, and Hamilton's 1982 short-time existence proof used his Nash–Moser theorem. Dennis DeTurck found in 1983 that composing with a suitable family of diffeomorphisms turns the equation into a strictly parabolic one, for which the ordinary contraction argument suffices (11A.4 Short-Time Existence and Uniqueness). The moral is a recurring one: before reaching for heavy machinery, look for a better choice of gauge.

Why Newton's method and smoothing work together is visible in Exercise 3.15: Newton's method squares the error at each step. Losing a fixed number of derivatives per step is a geometric loss; smoothing at a scale that improves with the step can absorb it, because the quadratic gain in accuracy outruns it. The simplified (contraction) iteration, with only a linear gain, cannot.

3.9History

Stefan Banach proved the contraction mapping principle in his 1922 thesis, the founding document of the theory of complete normed spaces; Picard had used successive approximations for ODEs in 1890, and Lindelöf's version dates from 1894. Maurice Fréchet introduced his derivative in 1911–1925, René Gâteaux his directional derivative in 1913, and the inverse and implicit function theorems in Banach spaces were standard by the 1930s through work of Hildebrandt and Graves (1927). Leonid Kantorovich proved his convergence theorem for Newton's method in 1948. Lyapunov (around 1906) and Erhard Schmidt (1908) introduced the reduction that bears their names in the study of rotating fluids and integral equations; Michael Crandall and Paul Rabinowitz proved their bifurcation theorem in 1971. Nash's embedding theorem appeared in 1956, Moser's abstraction in 1966, Hamilton's Nash–Moser theorem and his Ricci flow paper both in 1982, and DeTurck's trick in 1983.

3.10Exercises

Solution

(a) $\|x\|^2$ has derivative $2\langle x, \cdot\rangle$; compose with $t \mapsto \sqrt t$, differentiable at $\|x\|^2 > 0$, by the chain rule. (b) and (c). For any $h$ and small $t > 0$, $\|1 + th\|_\infty = 1 + t\max h$, and for small $t < 0$, $\|1 + th\|_\infty = 1 + |t|\max(-h) = 1 + t\min h$. So the one-sided directional derivatives are $\max h$ and $\min h$, which agree only for constant $h$: the norm is not even Gâteaux differentiable at $1$, hence not Fréchet differentiable. (In $L^p$, $1 < p < \infty$, the norm is Fréchet differentiable away from $0$; the corners of the sup norm are the failure.)

Solution

(a) $\frac1t(\sin(th) - th) \to 0$ pointwise with $|\frac1t(\sin th - th)| \leq 2|h|$, so in $L^2$ by dominated convergence. (b) $\sin h_\varepsilon = 0$ (since $\sin\pi = 0$), so the remainder is $-h_\varepsilon$, and the ratio is $1$ for every $\varepsilon$, while $\|h_\varepsilon\| = \pi\sqrt\varepsilon \to 0$. A Fréchet derivative would make the ratio tend to $0$.

Solution

$(A + H)^{-1} = (A(I + A^{-1}H))^{-1} = (I + A^{-1}H)^{-1}A^{-1} = \sum_{k\geq0}(-A^{-1}H)^kA^{-1}$ for $\|H\| < \|A^{-1}\|^{-1}$. The terms with $k \geq 2$ have norm at most $\frac{\|A^{-1}\|^3\|H\|^2}{1 - \|A^{-1}\|\|H\|} = O(\|H\|^2)$, leaving $A^{-1} - A^{-1}HA^{-1}$. For numbers, $\frac{d}{da}a^{-1} = -a^{-2}$.

Solution

$T^m$ has a unique fixed point $x^*$. Then $T^m(Tx^*) = T(T^mx^*) = Tx^*$, so $Tx^*$ is also a fixed point of $T^m$, hence $Tx^* = x^*$. Any fixed point of $T$ is one of $T^m$, so it is unique. For the Volterra operator $V$, $\|(V^nu)(s)\| \leq \frac{(|\lambda|Ks)^n}{n!}\|u\|_\infty$ by induction, so $\|V^n\| \leq \frac{(|\lambda|K)^n}{n!} < 1$ for large $n$; the affine map $u \mapsto f + Vu$ has $n$-th iterate $u \mapsto (\dots) + V^nu$, a contraction.

Solution

$x_{k + 1} - x^* = x_k - x^* - DF(x_k)^{-1}(F(x_k) - F(x^*)) = DF(x_k)^{-1}\big(F(x^*) - F(x_k) - DF(x_k)(x^* - x_k)\big)$. The bracket is the integral, of norm at most $\int_0^1Ls\|x^* - x_k\|^2ds = \frac L2\|x_k - x^*\|^2$. Multiply by $\beta$.

Solution

(a) $u(s) = (1 - s)\int_0^srf(r)\,dr + s\int_s^1(1 - r)f(r)\,dr$. Differentiating, $u'(s) = -\int_0^srf + \int_s^1(1 - r)f$ (the boundary terms cancel), and $u''(s) = -sf(s) - (1 - s)f(s) = -f(s)$; $u(0) = u(1) = 0$ is clear. (b) From the formulas, $|u| \leq \|f\|_\infty$, $|u'| \leq \|f\|_\infty$ and $|u''| = |f|$, so $\|u\|_{C^2} \leq 3\|f\|_\infty$. Injectivity: the only solution of $u'' = 0$ vanishing at both ends is $0$. So the operator is a bounded bijection with bounded inverse.

Solution

Given $(k, c)$, write $h = cx_0 + h_1$ with $h_1 \perp x_0$. Since $A$ is self-adjoint, $(A - \lambda_0)$ maps $x_0^\perp$ to itself and kills $x_0$. Pairing $(A - \lambda_0)h - \mu x_0 = k$ with $x_0$ gives $-\mu = \langle k, x_0\rangle$; the component orthogonal to $x_0$ gives $(A - \lambda_0)h_1 = k - \langle k, x_0\rangle x_0$, solvable uniquely. Differentiating $(A + \varepsilon B)x = \lambda x$: $Bx_0 + (A - \lambda_0)x'(0) = \lambda'(0)x_0$; pair with $x_0$ and use $\langle(A - \lambda_0)x', x_0\rangle = \langle x', (A - \lambda_0)x_0\rangle = 0$.

Solution

$\lambda u - u^3 = 0$ gives $u = 0$ or $u^2 = \lambda$: the trivial line and the parabola $\lambda = u^2$, a supercritical pitchfork. Here $L = D_uF(0, 0) = 0$, kernel $\mathbb{R}$, range $\{0\}$, and $D_\lambda D_uF = 1 \notin R$: transversality holds. For $\lambda u - u^2$: $u = 0$ or $u = \lambda$, two lines crossing, a transcritical bifurcation, again with transversality. In Crandall–Rabinowitz's terms, the branch is $\lambda(s) = s^2$ in the first case and $\lambda(s) = s$ in the second; the symmetry $u \mapsto -u$ of the first equation forces the even branch.