Stochastic Differential Equations and Feynman–Kac

A stochastic differential equation

describes a system that moves with velocity $b$ and is kicked by noise of size $\sigma$. It is an ODE with a random forcing, made rigorous by Itô's integral (6D.2 Martingales and the Itô Integral): it means $X_t = x + \int_0^tb(X_s)\,ds + \int_0^t\sigma(X_s)\,dB_s$. Its solutions, diffusions, model particles in fluids, interest rates, populations, neurons, and the noise schedules of generative AI.

The central fact of this chapter is that diffusions and second-order parabolic PDE are two descriptions of the same thing. To the SDE belongs the generator

and expectations of functions of the solution satisfy the backward Kolmogorov equation $u_t = Lu$, while the probability density of the solution satisfies the forward or Fokker–Planck equation $p_t = L^*p$, with the adjoint operator. With a killing rate or discount $r(x)$, the expectation $\mathbb{E}\big[e^{-\int r}g(X)\big]$ solves $u_t = Lu - ru$: the Feynman–Kac formula. That formula is how option prices are computed (6D.6 Black–Scholes), and it is the probabilistic meaning of every linear parabolic equation in this course.

By the end of this chapter you will be able to:

• prove existence and uniqueness for SDEs with Lipschitz coefficients by Picard iteration, and recognise non-uniqueness when the coefficients are not Lipschitz;
• compute the generator of a diffusion and use Dynkin's formula to find exit probabilities and expected exit times;
• prove the Feynman–Kac formula for classical solutions of $\partial_tu + Lu - ru = 0$;
• derive the Fokker–Planck equation, and find stationary distributions;
• solve the Ornstein–Uhlenbeck equation explicitly, and show that the Gibbs measure is stationary for Langevin dynamics.

3.1Running the noise backwards

3.2Existence and uniqueness

We work in $\mathbb{R}^n$ with an $m$-dimensional Brownian motion $B$ and coefficients $b : \mathbb{R}^n \to \mathbb{R}^n$, $\sigma : \mathbb{R}^n \to \mathbb{R}^{n\times m}$, satisfying the Lipschitz condition

which implies linear growth: $|b(x)| + |\sigma(x)| \leq K'(1 + |x|)$. (Time-dependent coefficients $b(t, x)$, $\sigma(t, x)$ are handled identically.)

The proof is the Picard–Lindelöf proof for ODEs (2B.9 The Inverse and Implicit Function Theorems) with the Itô isometry in place of the sup-norm estimate for an integral. As for ODEs, without the Lipschitz condition uniqueness can fail (Exercise 3.8), and without linear growth solutions can explode in finite time. Noise can also restore uniqueness: for $dX = b(X)\,dt + dB$ with $b$ merely bounded and measurable, Alexander Veretennikov proved in 1981 that the solution is unique, although the ODE $\dot x = b(x)$ may have many solutions.

The Markov property (statement; Øksendal, section 7.1). The solution started at $x$, observed from time $s$ on, is the solution started at $X_s$ driven by the increments of $B$ after $s$, which are independent of $\mathcal F_s$. Hence for bounded $g$,

and the same holds at stopping times (the strong Markov property).

3.3The generator and Dynkin's formula

For $f \in C^2$, Itô's formula (6D.2 Martingales and the Itô Integral) applied to $f(X_t)$, with $dX^i\,dX^j = a^{ij}(X)\,dt$, gives

The operator $L$ is the generator of the diffusion. For Brownian motion it is $\frac12\Delta$; for $dX = \mu\,dt + \sigma\,dB$ in one dimension it is $\mu\frac{d}{dx} + \frac{\sigma^2}{2}\frac{d^2}{dx^2}$. It is elliptic (possibly degenerate, since $a = \sigma\sigma^T \geq 0$), and every operator $b^i\partial_i + \frac12a^{ij}\partial_{ij}$ with $a \geq 0$ smooth arises this way (take $\sigma = a^{1/2}$).

Dynkin's formula turns questions about diffusions into boundary value problems. If $Lu = 0$ in a bounded domain $U$ with $u = g$ on $\partial U$ (and the exit time $\tau$ has finite mean), then $u(x) = \mathbb{E}_x[g(X_\tau)]$: Kakutani's theorem of 6D.1 Brownian Motion for general diffusions. If $Lv = -1$ in $U$ with $v = 0$ on $\partial U$, then $v(x) = \mathbb{E}_x\tau$. For example, Brownian motion with drift $\mu$ on an interval has exit probabilities given by the solutions $A + Be^{-2\mu x}$ of $\mu u' + \frac12u'' = 0$ (Exercise 3.9).

3.4The backward equation and Feynman–Kac

Three remarks. Uniqueness: the theorem shows that a bounded classical solution with bounded gradient is determined by $g$, a probabilistic uniqueness proof like 6A.3 The Heat Equation on ℝⁿ's maximum principle. Existence: conversely, if the coefficients and $g$ are smooth with bounded derivatives, the expectation is $C^{1,2}$ and solves the equation (statement; Øksendal, Theorem 8.2.1, and Friedman's books), so the formula constructs solutions; this is how Monte Carlo methods solve PDE (6D.10 Monte Carlo, Fourier and Finite Element Methods). Forward form: with $v(t, x) = u(T - t, x)$, the problem becomes the initial value problem $v_t = Lv - rv$, $v(0) = g$, and $v(t, x) = \mathbb{E}_x\big[e^{-\int_0^tr(X_s)ds}g(X_t)\big]$; for $r = 0$ this is the backward Kolmogorov equation $v_t = Lv$, and for Brownian motion it is the heat equation $v_t = \frac12\Delta v$ of 6A.3 The Heat Equation on ℝⁿ.

The formula is named after Richard Feynman, whose 1948 path-integral formulation of quantum mechanics expresses the Schrödinger propagator $e^{-itH}$ as an "integral over paths" weighted by $e^{iS/\hbar}$, and Mark Kac, who in 1949 saw that in imaginary time the path integral becomes a genuine integral over Brownian paths: the solution of $u_t = \frac12\Delta u - Vu$ is $\mathbb{E}_x\big[e^{-\int_0^tV(B_s)ds}g(B_t)\big]$. The potential $V$ acts as a killing rate: paths that spend time where $V$ is large are discounted. In finance $r$ is the interest rate and the formula says that the value of a contract is the discounted expected payoff.

3.5The forward equation

If $X_t$ (started at $x$) has a smooth density $p(t, y)$, then for $f \in C^\infty_c$, Dynkin's formula at the fixed time $t$ gives $\frac{d}{dt}\mathbb{E}f(X_t) = \mathbb{E}Lf(X_t)$, that is,

integrating by parts twice, where

is the formal adjoint. Since $f$ is arbitrary:

The two Kolmogorov equations are dual. The backward equation evolves observables ($u(t, x) = \mathbb{E}_xg(X_t)$, a function of the starting point), the forward equation evolves densities ($p(t, y)$, a function of the endpoint), and $\int u\,p_0 = \int g\,p_t$ pairs them. Adriaan Fokker (1914) and Max Planck (1917) wrote the forward equation for physical diffusions, and Andrei Kolmogorov (1931) gave both equations their general form, before stochastic integrals existed.

3.6The Ornstein–Uhlenbeck process

Leonard Ornstein and George Uhlenbeck introduced the process in 1930 to model the velocity of a Brownian particle, subject to friction ($-\theta X$) and random molecular kicks. In finance, Oldřich Vašíček used it in 1977 to model short-term interest rates, which are pulled back towards a long-run level (mean reversion): $dr = \theta(\bar r - r)\,dt + \sigma\,dB$. Its Fokker–Planck equation, $\partial_tp = \theta\partial_x(xp) + \frac{\sigma^2}{2}\partial_{xx}p$, is the equation whose Gaussian ground state was used in 6A.12 Entropy, Information and Diffusion.

3.7History

Andrei Kolmogorov derived the backward and forward equations for Markov processes in 1931. Kiyosi Itô's stochastic integral (1944) made it possible to construct diffusions path by path as solutions of SDEs, and Itô proved the existence and uniqueness theorem under Lipschitz conditions in the 1940s and early 1950s. Kac proved his formula in 1949, inspired by Feynman's path integrals (1948). Eugene Dynkin developed the theory of generators of Markov processes in the 1950s and 1960s. The Ornstein–Uhlenbeck process dates from 1930 and Vašíček's model from 1977.

3.8Exercises

Solution

Let $F(t) = \int_0^tf$. Then $F' = f \leq A + CF$ almost everywhere, so $(e^{-Ct}F)' \leq Ae^{-Ct}$ and $e^{-Ct}F(t) \leq \frac AC(1 - e^{-Ct})$, i.e. $CF(t) \leq A(e^{Ct} - 1)$. Then $f(t) \leq A + CF(t) \leq Ae^{Ct}$.

Solution

$\frac{d}{dt}t^3 = 3t^2 = 3(t^3)^{2/3}$. The coefficient $3x^{2/3}$ is not Lipschitz at $0$. For each $c \geq 0$, $X_t = 0$ for $t \leq c$ and $(t - c)^3$ for $t > c$ is a solution.

Solution

(a) Dynkin's formula with $Lu = 0$ gives the boundary value problem. Solutions are $A + Be^{-2\mu x}$; the boundary values give $u(x) = \frac{e^{-2\mu a} - e^{-2\mu x}}{e^{-2\mu a} - e^{-2\mu b}}$. (b) With $\mu = -0.05$, $-2\mu = 0.1$: $u(10) = \frac{1 - e^{1}}{1 - e^{3}} = \frac{1.718}{19.086} \approx 0.090$. A small drift against the gambler cuts the chance from $33\%$ to $9\%$ over this range: in the long run drift beats noise, because drift accumulates like $t$ and noise like $\sqrt t$.

Solution

(a) $\pi' = -\frac{2\theta x}{\sigma^2}\pi$, so $\frac{\sigma^2}{2}\pi' = -\theta x\pi$, and $\frac{\sigma^2}{2}\pi'' = -\theta(x\pi)'$; the sum is zero. (b) For $s < t$, $X_t = e^{-\theta(t - s)}X_s + \sigma\int_s^te^{-\theta(t - r)}dB_r$, and the integral is independent of $X_s$. So $\operatorname{Cov}(X_s, X_t) = e^{-\theta(t - s)}\operatorname{Var}X_s = \frac{\sigma^2}{2\theta}e^{-\theta(t - s)}$.

Solution

$w(s, x) = \mathbb{E}g(x + B_s)$ solves $w_s = \frac12\Delta w$ (6A.3 The Heat Equation on ℝⁿ, with $s$ in place of $2t$ and the probabilists' normalisation). Then $u(t, x) = e^{-r(T-t)}w(T - t, x)$ has $\partial_tu = ru - e^{-r(T-t)}w_s = ru - \frac12\Delta u$. It is the present value of a payment due at time $T$, when money earns interest at rate $r$.

Solution

$S_t = s\exp(\sigma B_t + (\mu - \frac12\sigma^2)t)$, so $\log S_t$ is as stated. The median of $S_t$ is $e^{\text{median of }\log S_t} = se^{(\mu - \sigma^2/2)t}$, and $\mathbb{E}S_t = se^{(\mu - \sigma^2/2)t}\mathbb{E}e^{\sigma B_t} = se^{(\mu - \sigma^2/2)t}e^{\sigma^2t/2} = se^{\mu t}$.

Solution

(a) $\frac{d}{dt}\int uv\,dV = \int(\partial_tu\,v + u\,\partial_tv - Ruv)\,dV = \int(v\Delta u - u\Delta v + Ruv - Ruv)\,dV = 0$, integrating by parts on the closed manifold. (b) The heat equation evolves observables and the conjugate equation evolves densities with respect to the moving volume $dV_t$; the pairing $\int uv\,dV_t$ is conserved, as $\int u\,p$ is for the Kolmogorov pair. The term $Rv$ compensates the shrinking of the volume form, so that $v\,dV_t$ evolves as a measure. (c) Then $v\,dV$ behaves like the law of a diffusion running backwards in time, which is why Perelman's reduced volume and entropy can be read as quantities about Brownian motion in space-time.