Calculus in Several Variables

Course 1 met derivatives of several variables as computations: gradients, Jacobian matrices, Hessians (1A.7 The Derivative as a Linear Approximation, 1A.8 Gradient, Jacobian and Hessian). This chapter makes them rigorous. The central definition is the one stated informally in Course 1: the derivative of a map at a point is the best linear approximation to the map near that point. In one variable, a linear map is multiplication by a number, so the derivative could be treated as a number. In several variables it is a linear map, a matrix, and treating it as one is what makes everything work.

The second half of the chapter is about second derivatives. The Hessian, the matrix of second partial derivatives, is symmetric under a mild hypothesis, and so it is a quadratic form: the spectral theorem of 1A.6 Symmetric Matrices and the Spectral Theorem diagonalises it, and its eigenvalues decide whether a critical point is a peak, a pit or a pass. This is the first link of thread Q, which leads to the Ricci tensor. And at an interior maximum, the Hessian is negative semidefinite, so its trace, the Laplacian, is at most zero. That single observation is the seed of every maximum principle for the heat equation, and of Hamilton's maximum principle for Ricci flow.

By the end of this chapter you will be able to:

• define the derivative of a map between Euclidean spaces as a linear map, and compute it as the Jacobian matrix;
• give examples where partial derivatives exist but the function isn't differentiable, and prove that continuous partials are enough;
• prove and use the chain rule and the mean value inequality;
• prove Clairaut's theorem on mixed partials, and write the second-order Taylor expansion with the Hessian;
• classify critical points by the eigenvalues of the Hessian, and prove that $\Delta u \leq 0$ at an interior maximum.

8.1Propagating uncertainty

8.2Linear maps and their size

A linear map $L : \mathbb{R}^n \to \mathbb{R}^m$ is given by an $m \times n$ matrix. Its operator norm

is the largest factor by which $L$ stretches a vector, so $|Lv| \leq \|L\|\,|v|$ for every $v$. It is finite: if $L$ has entries $a_{ij}$, Cauchy–Schwarz gives $|Lv| \leq \big(\sum_{ij}a_{ij}^2\big)^{1/2}|v|$. Hence every linear map between Euclidean spaces is Lipschitz, and so continuous. The operator norm is a norm on the space of matrices, and it satisfies $\|LM\| \leq \|L\|\,\|M\|$, which was used to make the matrix exponential converge in 2B.6 Power Series, Exponentials and Bump Functions. (In infinite dimensions linear maps can be unbounded, and that is the starting point of functional analysis, 4A.1 Banach Spaces and Bounded Operators.)

8.3The derivative

Throughout, $U$ is an open subset of $\mathbb{R}^n$, and $|\cdot|$ is the Euclidean norm.

The error $f(x_0 + h) - f(x_0) - Lh$ must be small compared with $|h|$, as $h \to 0$ from every direction at once. The derivative is unique: if $L_1$ and $L_2$ both work, then $|(L_1 - L_2)h|/|h| \to 0$; taking $h = tv$ with $t \to 0$ gives $(L_1 - L_2)v = 0$ for every $v$. Differentiable maps are continuous at $x_0$, since $|f(x_0 + h) - f(x_0)| \leq |Lh| + o(|h|) \to 0$.

8.3.1Partial and directional derivatives

The directional derivative of $f$ at $x_0$ in direction $v$ is $D_vf(x_0) = \lim_{t\to 0}\frac{f(x_0 + tv) - f(x_0)}{t}$, if it exists. The partial derivatives $\partial_jf = \frac{\partial f}{\partial x_j}$ are the directional derivatives along the coordinate directions $e_j$. If $f$ is differentiable, then $D_vf(x_0) = Df(x_0)v$ (take $h = tv$ in the definition), so the matrix of $Df(x_0)$ has the partial derivatives $\partial_jf_i(x_0)$ as its entries: it is the Jacobian matrix.

The converse fails, and badly.

What rescues the situation is continuity of the partials.

A map whose partial derivatives exist and are continuous on $U$ is called continuously differentiable, or $C^1$. In practice nearly every map we differentiate is $C^1$, and the theorem says that for them the Jacobian matrix is the derivative.

8.4The chain rule and the mean value inequality

Two consequences are used constantly.

The gradient is perpendicular to level sets. For real-valued $f$, the derivative is a row vector, and its transpose is the gradient $\nabla f$, so that $Df(x_0)v = \nabla f(x_0)\cdot v$. If $\gamma(t)$ is a curve in a level set $\{f = c\}$ with $\gamma(0) = x_0$, then $f(\gamma(t)) = c$, and the chain rule gives $\nabla f(x_0)\cdot\gamma'(0) = 0$. Among unit vectors $v$, $\nabla f\cdot v$ is largest for $v = \nabla f/|\nabla f|$ (Cauchy–Schwarz), so the gradient points in the direction of steepest ascent. 2B.9 The Inverse and Implicit Function Theorems shows that, when $\nabla f(x_0) \neq 0$, the level set really is a smooth hypersurface near $x_0$.

There is no mean value equality for vector-valued maps (the circle $t \mapsto (\cos t, \sin t)$ returns to its start, but its derivative never vanishes), and the inequality is the correct replacement. It is what turns a bound on $Df$ into a Lipschitz bound, and so into a contraction, in the proof of the inverse function theorem (2B.9 The Inverse and Implicit Function Theorems). It also proves the fact promised in 2B.4 Connectedness: a map with $Df = 0$ on a connected open set is constant. On each ball it is constant by the inequality, so the set where $f$ equals its value at a chosen point is open, and it is closed by continuity.

8.5Second derivatives

If the partial derivatives $\partial_jf$ are themselves differentiable, we get second partials $\partial_i\partial_jf$. A function whose partials up to order $k$ exist and are continuous is $C^k$, and $C^\infty$ (smooth) if this holds for every $k$.

The second difference $\Delta(h)$ treats the two variables symmetrically, which is why the order doesn't matter in the limit. Without continuity it can matter: for $f(x, y) = \frac{xy(x^2 - y^2)}{x^2 + y^2}$ (and $f(0) = 0$), $\partial_x\partial_yf(0, 0) = 1$ but $\partial_y\partial_xf(0, 0) = -1$ (Exercise 8.14).

8.6Critical points and the Hessian

A critical point of $f$ is a point where $\nabla f = 0$. At a local maximum or minimum in the interior of $U$, every directional derivative vanishes (the one-variable test of 2A.10 Derivatives along each line), so local extrema are critical points. Which critical points are extrema is decided by the Hessian.

Because $\nabla^2f(x_0)$ is symmetric, the spectral theorem (1A.6 Symmetric Matrices and the Spectral Theorem) gives an orthonormal basis of eigenvectors $v_1, \dots, v_n$ with real eigenvalues $\lambda_1, \dots, \lambda_n$. In these coordinates, at a critical point, the Taylor expansion reads

The number of negative eigenvalues of the Hessian is the index of the critical point: $0$ for a minimum, $n$ for a maximum, in between for saddles. The quantity is robust: a small perturbation of $f$ moves a non-degenerate critical point slightly but doesn't change its index. That robustness is the basis of Morse theory (7A.10 Morse Theory), which reads the topology of a space off the indices of the critical points of a function on it.

8.7The Laplacian at a maximum

Here is the observation that this chapter contributes to thread M.

The statement is elementary, and its consequences are not. If $u$ solves the heat equation $\partial_tu = \Delta u$, then at a point where $u(\cdot, t)$ is largest, $\partial_tu = \Delta u \leq 0$: the maximum can't increase. Made rigorous with Hamilton's trick from 2A.10 Derivatives, this is the weak maximum principle for the heat equation (6A.4 Maximum Principles). On a ring, 2B.7 Fourier Series and the First Heat Equation proved it from a formula; this argument needs no formula, and works on any domain and, with the Laplacian of a Riemannian metric, on any closed manifold.

8.8History

The symmetry of mixed partial derivatives was used by Euler and by Clairaut in the 1730s and 1740s; Hermann Amandus Schwarz gave a rigorous proof in 1873, and the counterexample $\frac{xy(x^2-y^2)}{x^2+y^2}$ appeared in Giuseppe Peano's 1884 edition of Angelo Genocchi's calculus lectures. The modern definition of the derivative as a linear approximation appears in Otto Stolz's calculus text of 1893 and was championed by W. H. Young (1910) and Maurice Fréchet (1911), whose name it carries in infinite dimensions. Its advantage, that it treats all directions at once and makes the chain rule a product of matrices, is why every later book in this guidebook uses it.

8.9Exercises

Solution

(a) $\begin{pmatrix}2x & -2y\\ 2y & 2x\end{pmatrix}$. (b) $\begin{pmatrix}\cos\theta & -r\sin\theta\\ \sin\theta & r\cos\theta\end{pmatrix}$, with determinant $r$: singular at $r = 0$. (c) $2x^{\mathsf T}$. (d) $x^{\mathsf T}/|x|$.

Solution

For $v = (a, b)$ with $b \neq 0$: $g(ta, tb)/t = \frac{a^2b}{t^2a^4 + b^2} \to \frac{a^2}{b}$. For $b = 0$, $g(ta, 0) = 0$, so $D_vg(0) = 0$. On $y = x^2$: $\frac{x^4}{2x^4} = \frac12$.

Solution

For $y \neq 0$, $\partial_xf(0, y) = \lim_{x\to0}\frac{f(x, y)}{x} = \lim\frac{y(x^2 - y^2)}{x^2 + y^2} = -y$; also $\partial_xf(0, 0) = 0$. Similarly $\partial_yf(x, 0) = x$. So $\partial_y(\partial_xf)(0, 0) = -1$ and $\partial_x(\partial_yf)(0, 0) = 1$. The second partials exist but are not continuous at $0$.

Solution

$\nabla f = (3x^2 - 3y^2, -6xy)$ and $\nabla^2f = \begin{pmatrix}6x & -6y\\ -6y & -6x\end{pmatrix}$, both $0$ at the origin. In polar coordinates $f = r^3\cos 3\theta$, which changes sign six times as $\theta$ goes round.

Solution

$g = 4\pi^2/2.006^2 \approx 9.811$ m/s². The relative sensitivities are $1$ for $L$ and $-2$ for $T$, so $\frac{u(g)}{g} = \sqrt{(0.002)^2 + (2 \times 0.004/2.006)^2} \approx \sqrt{4\times10^{-6} + 1.59\times10^{-5}} \approx 0.0045$, so $u(g) \approx 0.044$ m/s². The period contributes about four times as much variance, because it enters squared.

Solution

(a) $\partial_if = x_i/2$, $\partial_i\partial_jf = \tfrac12\delta_{ij}$, trace $n/2$. (b) $|\nabla f|^2 = |x|^2/4 = f$, and $2\cdot\frac n2 - f + f - n = 0$. (c) The integral factors into $n$ one-dimensional integrals, each $(4\pi)^{-1/2}\sqrt{4\pi} = 1$.