Banach Spaces and Bounded Operators

Book 3A produced spaces of functions, $L^p$ and $C(K)$, that are vector spaces with a notion of length, and complete. This book studies such spaces in general. The subject is functional analysis, and its slogan is that functions are points: a solution of a differential equation is a point in a space of functions, an equation is a map between such spaces, and solving it is a question about that map. Linear algebra handles this in finite dimensions (1A.2 Linear Maps and Matrices). Functional analysis does it in infinite dimensions, where the most useful facts of linear algebra, above all compactness of the unit ball, fail.

This first chapter sets up the language: normed spaces and Banach spaces (complete normed spaces), bounded linear operators between them, and the operator norm. Then it proves the theorem that explains why the rest of the book is needed: the closed unit ball of a normed space is compact if and only if the space is finite-dimensional (Riesz's lemma). In infinite dimensions, bounded sequences need not have convergent subsequences, and every later chapter is, in one way or another, about getting compactness back. The chapter ends with the Neumann series, the infinite-dimensional geometric series, which inverts operators close to the identity and is the simplest perturbative existence theorem.

By the end of this chapter you will be able to:

• recognise the standard Banach spaces ($\ell^p$, $L^p$, $C(K)$, $C^k$) and show that a normed space is or isn't complete;
• prove that a linear map is continuous exactly when it is bounded, and compute operator norms;
• prove Riesz's lemma and deduce that the unit ball is compact only in finite dimensions;
• invert $I - A$ by the Neumann series, and show that invertible operators form an open set;
• interpret operator norms and condition numbers in practice.

1.1A vibrating string

1.2Normed and Banach spaces

The examples so far are all Banach spaces:

• $\mathbb{R}^n$ and $\mathbb{C}^n$ with any norm (2B.3 Compactness).
• $L^p(X, \mu)$ for $1 \leq p \leq \infty$, and the sequence spaces $\ell^p$ (3A.7 Lᵖ Spaces and Jensen’s Inequality).
• $C(K)$, continuous functions on a compact metric space, with the sup norm (2B.5 Uniform Convergence and Arzelà–Ascoli).
• $C^k([a, b])$, $k$ times continuously differentiable functions, with $\|f\|_{C^k} = \sum_{j=0}^k\sup|f^{(j)}|$. Completeness follows from 2B.5 Uniform Convergence and Arzelà–Ascoli's theorem on uniform limits of derivatives.
• Hölder spaces $C^{k,\alpha}$ (4A.11 Hölder Spaces) and Sobolev spaces $W^{k,p}$ (4A.9 Sobolev Spaces), later in this book.

The norm has to fit the space. $C^1([0, 1])$ with the sup norm (ignoring the derivative) is a normed space but not a Banach space: smooth functions such as $\sqrt{(x - \frac12)^2 + \frac1n}$ converge uniformly to $|x - \frac12|$, which is not $C^1$. The sequence is Cauchy in the sup norm, and its limit is outside the space. Complete spaces come from norms that control everything the space's definition asks for.

A useful test: a normed space is complete if and only if every absolutely convergent series converges, that is, $\sum\|x_k\| < \infty$ implies that the partial sums of $\sum x_k$ converge (Exercise 1.9). That is how Riesz–Fischer was proved in 3A.7 Lᵖ Spaces and Jensen’s Inequality.

1.3Bounded linear operators

A linear map between normed spaces is usually called an operator.

"Bounded" here means bounded on the unit ball, not bounded on the whole space (no non-zero linear map is that). The best constant is the operator norm

the largest factor by which $T$ stretches a vector (2B.8 Calculus in Several Variables). The bounded operators from $X$ to $Y$ form a vector space $L(X, Y)$, normed by $\|T\|$, and $\|ST\| \leq \|S\|\,\|T\|$. If $Y$ is a Banach space, so is $L(X, Y)$ (Exercise 1.12). The case $Y = \mathbb{R}$ (or $\mathbb{C}$) gives the dual space $X^* = L(X, \mathbb{R})$ of bounded linear functionals, the subject of 4A.3 Hahn–Banach and Duality.

1.4Finite dimensions are special

In a finite-dimensional normed space everything is as in $\mathbb{R}^n$. All norms are equivalent (2B.3 Compactness, proved there by the contradiction–compactness template); hence every finite-dimensional normed space is complete, every finite-dimensional subspace of a normed space is closed, every linear map from a finite-dimensional space is bounded, and closed bounded sets are compact (Heine–Borel). The last property is the one that characterises finite dimensions.

This is the theorem behind the rest of the book. Bounded sequences in infinite-dimensional spaces need not have convergent subsequences, so existence proofs can't simply extract a limit as in 2B.3 Compactness. Compactness has to be recovered in other ways: in a weaker topology (4A.6 Weak Convergence and the Direct Method), by an extra derivative bound that makes a family equicontinuous (2B.5 Uniform Convergence and Arzelà–Ascoli) or precompact in a Sobolev sense (4A.10 Sobolev Embeddings and Critical Exponents), or by operators that are themselves compact (4A.7 Compact Operators and Spectra).

1.5The Neumann series

In a Banach algebra of operators, the geometric series of 2A.7 Series inverts operators close to the identity.

The equation $(I - A)u = f$ is solved by $u = f + Af + A^2f + \cdots$, the iteration $u_{k+1} = f + Au_k$, which is a contraction (2B.2 Completeness and Contraction) because $\|A\| < 1$. The Neumann series is the linear case of the contraction mapping principle, with a formula.

1.6History

Stefan Banach's 1920 thesis and his book Théorie des opérations linéaires (1932) made complete normed spaces a subject; Norbert Wiener and Hans Hahn reached the same definition independently around 1920–22. Frigyes Riesz proved his lemma in 1918, in a paper on what are now called compact operators. The series inversion goes back to Carl Neumann's work on potential theory in 1877, and Ivar Fredholm's 1903 theory of integral equations, the first great success of infinite-dimensional linear algebra, used it. Vito Volterra studied the integral equations named after him in the 1890s.

1.7Exercises

Solution

$|f_n(x) - |x - \tfrac12|| \leq \frac{1}{\sqrt n}$, so $f_n$ converges uniformly to $|x - \frac12|$, hence is Cauchy in the sup norm. The derivatives $f_n'(x) = \frac{x - 1/2}{\sqrt{(x-1/2)^2 + 1/n}}$ converge pointwise to $\operatorname{sign}(x - \frac12)$, which is discontinuous, so they don't converge uniformly and $(f_n)$ isn't Cauchy in $C^1$.

Solution

$L_t = L_0(I + L_0^{-1}(L_t - L_0))$ and $\|L_0^{-1}(L_t - L_0)\| \leq CKt < 1$. By the Neumann series, $\|L_t^{-1}\| \leq \frac{\|L_0^{-1}\|}{1 - CKt}$.