Ordinary Differential Equations

An ordinary differential equation (ODE) says how a quantity changes in terms of its current value: $y'(t) = F(t, y(t))$. Population growth, radioactive decay, the motion of planets, chemical reactions and electrical circuits are all ODEs. They are not in Tao's two volumes, and none of the Path's books before Course 8 treat them carefully, yet every later course uses them: the flow of a vector field on a manifold (8A.6 Flows and the Lie Derivative), geodesics (9A.3 Geodesics and the Exponential Map), and the evolution of curvature under Ricci flow, which at each point behaves like an ODE plus a diffusion term (11A.2 How Curvature Evolves, 11A.4 Maximum Principles under Ricci Flow). So the guidebook does them here, where the tools are ready.

The chapter answers four questions. Do solutions exist, and are they unique? Yes, if $F$ is Lipschitz, by the contraction principle of 2B.2 Completeness and Contraction applied in the function space $C([0, T])$ of 2B.5 Uniform Convergence and Arzelà–Ascoli. How sensitive are they to the initial data? At most exponentially, by Gronwall's inequality, and sometimes exactly that sensitive, which is why weather can't be forecast indefinitely. How long do they last? Either for ever or until they become infinite: the blow-up alternative, the ODE form of the statement that a Ricci flow exists as long as its curvature stays bounded. What regions do they stay in? Those whose boundary the vector field doesn't cross outwards: invariant sets, the ODE fact behind Hamilton's maximum principle for systems (11A.4 Maximum Principles under Ricci Flow).

By the end of this chapter you will be able to:

• prove the Picard–Lindelöf existence and uniqueness theorem by contraction;
• prove Gronwall's inequality and use it for uniqueness and continuous dependence on initial data;
• use the blow-up alternative to turn an a-priori bound into global existence, and compute blow-up times;
• solve linear systems with the matrix exponential and read stability from eigenvalues;
• prove comparison principles for scalar ODEs and invariance of convex sets for systems, and check the hypotheses in examples.

10.1How far ahead can we predict?

10.2Existence and uniqueness

We study the initial value problem

where $y$ takes values in $\mathbb{R}^n$ and $F$ is continuous on an open set of $\mathbb{R} \times \mathbb{R}^n$. Higher-order equations reduce to this: $u'' = -u$ becomes the system $(u, v)' = (v, -u)$.

By the fundamental theorem of calculus (2A.11 The Riemann Integral), a continuous function $y$ solves (IVP) on an interval if and only if it solves the integral equation

The integral form needs no derivatives, and it is the one that sets up a fixed-point problem.

Two examples show what can go wrong.

• Non-uniqueness. $y' = 3y^{2/3}$, $y(0) = 0$ has the solution $y = 0$, and also $y = t^3$, and also, for any $c > 0$, the function that is $0$ until $t = c$ and $(t - c)^3$ afterwards. The function $3y^{2/3}$ is continuous but has infinite slope at $0$.
• Finite-time blow-up. $y' = y^2$, $y(0) = 1$ has the solution $y = \frac{1}{1 - t}$, which becomes infinite at $t = 1$ (2A.10 Derivatives). The right-hand side is perfectly smooth, but it grows faster than linearly.

The first problem is cured by a Lipschitz condition. The second is not a problem to be cured but a phenomenon to be understood.

The iterates $\Phi^k(y_0)$, starting from the constant function, are the Picard iterates. For $y' = y$ they are the Taylor polynomials of $e^t$ (2B.2 Completeness and Contraction). The same theorem holds backwards in time, on $[t_0 - T, t_0]$.

A second, quite different existence theorem is worth knowing. Peano's theorem says that if $F$ is merely continuous, solutions still exist (but may not be unique, as $3y^{2/3}$ showed). The proof is by compactness instead of contraction: build approximate solutions by Euler's method (straight-line steps), note that they all have slope at most $M$, so they are equicontinuous, and extract a uniformly convergent subsequence by Arzelà–Ascoli (2B.5 Uniform Convergence and Arzelà–Ascoli). The limit solves the integral equation (Exercise 10.13). The two proofs are the two engines of existence, completeness and compactness, side by side.

10.3Gronwall's inequality

The proof is the comparison argument of 2A.10 Derivatives: a quantity growing no faster than $L$ times itself grows no faster than $e^{Lt}$. Its consequences are the basic stability facts about ODEs.

For the Lorenz system, a Lipschitz constant $L$ on the region where solutions live gives the bound $e^{Lt}$ on the growth of errors. The bound is crude (here $L$ is much larger than the observed rate $0.9$), but the form of the bound, exponential, is attained. No better general statement is possible.

Differentiability in the initial data also holds. If $F$ is $C^1$, the solution $y(t; y_0)$ is $C^1$ in $y_0$, and its derivative $J(t) = \partial y/\partial y_0$ solves the linearised (or variational) equation $J' = D_yF(t, y(t))\,J$ with $J(t_0) = I$. (Differentiate the equation in $y_0$; the proof that this is legitimate is an application of the implicit function theorem or of Gronwall to difference quotients, and we take it on trust.) The growth rate of $J$ along a typical solution of the Lorenz system is its Lyapunov exponent, about $0.9$.

10.4How long do solutions last?

Picard–Lindelöf gives existence for a short time. Gluing local solutions together, using uniqueness to make the pieces agree, gives a maximal solution, defined on a largest open interval $(T_-, T_+)$ containing $t_0$, and the question is what happens at its ends.

The theorem turns the existence question into a bounds question. If you can prove an a-priori bound, that every solution stays in some compact set on every finite time interval, then solutions exist for all time. For example, if $|F(y)| \leq A + B|y|$ (linear growth), Gronwall applied to $|y(t)| \leq |y_0| + \int(A + B|y|)$ shows $|y|$ can't blow up in finite time, so solutions are global. In particular every linear system $y' = A(t)y + b(t)$ with continuous coefficients has solutions for all time.

10.5Linear systems

For a constant matrix $A$, the solution of $y' = Ay$ is $y(t) = e^{tA}y_0$ (2B.6 Power Series, Exponentials and Bump Functions). Its long-time behaviour is decided by the eigenvalues of $A$. If $A$ is diagonalisable with eigenvalues $\lambda_i$ and eigenvectors $v_i$, then writing $y_0 = \sum c_iv_i$,

Components along eigenvalues with negative real part decay, those with positive real part grow, and complex eigenvalues $a \pm ib$ produce rotation at rate $b$ with growth or decay $e^{at}$ (2B.6 Power Series, Exponentials and Bump Functions). In the plane this gives the standard phase portraits: a stable node (two negative eigenvalues), an unstable node, a saddle (eigenvalues of opposite signs, with a stable and an unstable direction), spirals and centres. The equilibrium $0$ is asymptotically stable exactly when every eigenvalue has negative real part.

The linear theory describes nonlinear systems near equilibria. If $F(y^*) = 0$, then near $y^*$, $y' = F(y) \approx DF(y^*)(y - y^*)$, and the eigenvalues of $DF(y^*)$ decide stability: if all have negative real part, $y^*$ is asymptotically stable for the nonlinear system too (Exercise 10.12). This is the "linearise" step of thread L applied to dynamics. In 11B.1 Ricci Solitons the same question, stability of an equilibrium, is asked of Ricci solitons, which are the equilibria of Ricci flow up to scaling and diffeomorphism.

10.6Comparison and invariant sets

10.6.1The comparison principle

For scalar equations, solutions can't cross. That gives a powerful way to control a solution of a complicated equation by one of a simpler equation.

In 2A.10 Derivatives this was done by hand for $F(y) = cy^2$, to show that scalar curvature forces a singularity. The general version says: to bound a quantity that satisfies a differential inequality, solve the corresponding equation and compare. In PDE the same idea, applied at the maximum point of a function, becomes the maximum principle (6A.4 Maximum Principles).

10.6.2Invariant sets

For systems there is no ordering of $\mathbb{R}^n$ to compare with, and the right notion is an invariant set: a set $K$ such that solutions starting in $K$ stay in $K$. When is a closed set invariant? Intuitively, when the vector field never points out of it at its boundary. For convex sets this has a clean form. A vector $\nu$ is an outward normal to a closed convex set $K$ at a boundary point $p$ if $\nu\cdot(q - p) \leq 0$ for every $q \in K$: the whole of $K$ lies on one side of the plane through $p$ perpendicular to $\nu$.

10.7History

Newton and Leibniz solved the first differential equations in the 1670s–1690s, and through the 18th century Euler and others solved many by explicit formulas. Cauchy, in lectures from the 1820s, gave the first existence proof, by the polygon (Euler) approximations. Rudolf Lipschitz introduced his condition in 1876; Émile Picard developed successive approximations in 1890, and Ernst Lindelöf gave the theorem its modern form in 1894. Giuseppe Peano proved existence for continuous right-hand sides in 1886 and 1890. Thomas Grönwall published his inequality in 1919. Henri Poincaré, from the 1880s, turned the subject from finding formulas to describing the shapes of solutions, the qualitative theory of which phase portraits and invariant sets are part, and Lorenz's 1963 paper showed how strange those shapes can be. William Kermack and Anderson McKendrick's epidemic model appeared in 1927. Hamilton's use of invariant sets for the curvature ODE dates from his 1982 and 1986 papers on 3- and 4-manifolds.

10.8Exercises

Solution

$y_1 = t$, $y_2 = t + \frac{t^3}{3}$, $y_3 = t + \frac{t^3}{3} + \frac{2t^5}{15} + \frac{t^7}{63}$. The Taylor series of $\tan t$ is $t + \frac{t^3}{3} + \frac{2t^5}{15} + \frac{17t^7}{315} + \cdots$: the iterates agree with it up to $t^5$. The solution exists on $(-\pi/2, \pi/2)$ and $|\tan t| \to \infty$ at the ends.

Solution

Separating variables, $t = \int_{y_0}^{y(t)}\frac{ds}{F(s)}$; $y$ is increasing, and $y(t) \to \infty$ as $t$ approaches $\int_{y_0}^\infty\frac{ds}{F(s)}$, which is the existence time. $y^{1+\alpha}$: $T = \frac{y_0^{-\alpha}}{\alpha}$. $y\log y$: $\int\frac{dy}{y\log y} = \log\log y \to \infty$, so no blow-up (the solution is $\exp(e^t\log y_0)$, growing doubly exponentially). $e^y$: $T = e^{-y_0}$.

Solution

(a) $|y - \pi(y) - s(q - \pi(y))|^2 = |y - \pi(y)|^2 - 2s(y - \pi(y))\cdot(q - \pi(y)) + s^2|q - \pi(y)|^2$ must be $\geq |y - \pi(y)|^2$ for $s \in (0, 1]$; divide by $s$ and let $s \to 0$. (b) Apply (a) at $y$ with $q = \pi(z)$ and at $z$ with $q = \pi(y)$, add, and use Cauchy–Schwarz: $|\pi(y) - \pi(z)|^2 \leq (y - z)\cdot(\pi(y) - \pi(z))$. (c) The upper bound is $d(z) \leq \tfrac12|z - \pi(y)|^2$ expanded. For the lower bound, the same expansion at $z$ gives $d(y) \leq d(z) + (z - \pi(z))\cdot(y - z) + \tfrac12|y - z|^2$, and $|(z - \pi(z)) - (y - \pi(y))| \leq 2|z - y|$ by (b). So $|d(z) - d(y) - (y - \pi(y))\cdot(z - y)| \leq \tfrac52|z - y|^2$.

Solution

(a) At a boundary point $p \in K$, some coordinates vanish, and the outward normals are the vectors $-(a, b, c)$ with $a, b, c \geq 0$, non-zero only in the vanishing coordinates. On $K$ every component of $F$ is a sum of products of non-negative numbers, so $F(p) \in K$ and the inner product with any outward normal is $\leq 0$. (b) $\lambda^2 + \mu^2 + \nu^2 + \lambda\mu + \lambda\nu + \mu\nu = \tfrac12\big((\lambda + \mu)^2 + (\lambda + \nu)^2 + (\mu + \nu)^2\big) \geq 0$. (c) At $(\lambda, \mu, \nu) = (2, -1, -1)$, $\nu' = 1 + 2\cdot(-1) = -1 < 0$, so the solution immediately enters $\{\nu < -1\}$.