Proof of Comparison Principle for Entropy-Penalised Viscosity Solutions of the Discounted Hamilton-Jacobi-Bellman Equation on the Torus Wasserstein Space

Each result cited below is universally quantified over the data in its own statement.

We write $\mathcal{P}$, $\mathcal{P}^{\mathrm{ac}}$, $\mathcal{P}^{\mathrm{Ent}}$ and $\mathcal{P}^{\mathcal{I}}$ for $\mathcal{P}(\mathbb{T}^{d})$, $\mathcal{P}^{\mathrm{ac}}(\mathbb{T}^{d})$, $\mathcal{P}^{\mathrm{Ent}}(\mathbb{T}^{d})$ and $\mathcal{P}^{\mathcal{I}}(\mathbb{T}^{d})$, and $W$ for $W_{\mathbb{T}}$. For $\nu\in\mathcal{P}$ let $\Phi_{\nu}(\mu)=\tfrac12W(\mu,\nu)^{2}$.

Step 0 (Facts used repeatedly). (F1) $W$ is a metric on $\mathcal{P}$ by The Torus Wasserstein Space is a Sequentially Compact Metric Space in which Optimal Couplings Exist §metric: it is symmetric, satisfies the triangle inequality, $W(\mu,\mu)=0$, and $W(\mu,\nu)>0$ for $\mu\ne\nu$. (F2) For all $\mu,\nu\in\mathcal{P}$ one has $W(\mu,\nu)^{2}\le d/4$, because every torus cost lies in $[0,d/4]$ by Probability Measures on the Flat Torus, the Torus Cost of a Coupling, the Torus Wasserstein Distance and Optimal Couplings §cost and $W(\mu,\nu)^{2}$ is the infimum of such costs by Probability Measures on the Flat Torus, the Torus Cost of a Coupling, the Torus Wasserstein Distance and Optimal Couplings §distance. (F3) Let $\mu\in\mathcal{P}^{\mathrm{ac}}$, $\nu\in\mathcal{P}$, and let $T$ be an optimal map from $\mu$ to $\nu$, which exists by McCann's Theorem on the Flat Torus: Optimal Couplings out of an Absolutely Continuous Measure are Induced by a Unique Map with a Periodic Potential §map. Then $\Phi_{\nu}$ is an intrinsic test function on $\mathcal{P}^{\mathrm{ac}}$ with $\nabla\Phi_{\nu}(\mu)=-v_{T}$ by Half the Squared Torus Wasserstein Distance to a Fixed Measure is an Intrinsic Test Function on the Absolutely Continuous Measures §test, and $W(\mu,\nu)^{2}=2\Phi_{\nu}(\mu)=\lVert v_{T}\rVert_{\mu}^{2}$ by Half the Squared Torus Wasserstein Distance to a Fixed Measure: the One-Sided Bound Along Couplings, Differentiability at an Absolutely Continuous Source, and the Tangent Gradient §upper; hence $\lVert v_{T}\rVert_{\mu}=W(\mu,\nu)\le\sqrt d/2$ by (F2). (F4) Let $\mu\in\mathcal{P}^{\mathrm{ac}}$ and let $T_{0}$ be an optimal map from $\mu$ to $\mu$. By (F3) and (F1), $\lVert v_{T_{0}}\rVert_{\mu}^{2}=W(\mu,\mu)^{2}=0$, so $v_{T_{0}}=0$ in $L^{2}(\mu;\mathbb{R}^{d})$ and $\nabla\Phi_{\mu}(\mu)=0$. (F5) Every member of $\mathcal{P}^{\mathrm{Ent}}$ is absolutely continuous and has nonnegative entropy, by The Entropy of Measures on the Torus: Nonnegativity, Absolute Continuity and Closed Sublevel Sets §nonnegative.

Step 1 (Contradiction hypothesis and constants). Suppose, for a contradiction, that $m:=u(\mu_{0})-v(\mu_{0})>0$ for some $\mu_{0}\in\mathcal{P}^{\mathrm{Ent}}$; in particular $\mathcal{P}$ is nonempty. By The Torus Wasserstein Space is a Sequentially Compact Metric Space in which Optimal Couplings Exist §compact, $\mathcal{P}$ is sequentially compact in $(\mathcal{P},W)$, hence compact by A Sequentially Compact Subset of a Metric Space is Compact. By claim 1 (Maximum) and claim 2 (Minimum) of Semicontinuous Functions Attain Their Extrema on a Compact Set, applied to the upper semicontinuous $u$ and the lower semicontinuous $v$, the real numbers $U=\max_{\mathcal{P}}u$ and $V=\min_{\mathcal{P}}v$ exist. Let $\delta_{u},c_{u}$ be the number and nondecreasing function of the subsolution property of $u$, and $\delta_{v},c_{v}$ those of the supersolution property of $v$. Put $\delta_{0}=\min\{\delta_{u},\delta_{v}\}>0$ and $c(t)=\max\{c_{u}(t),c_{v}(t)\}$ for $t\ge0$; then $c:[0,\infty)\to[0,\infty)$ is nondecreasing and $c_{u}\le c$, $c_{v}\le c$.

Step 2 (The doubled functionals). For real $\varepsilon>0$ and $\mu,\nu\in\mathcal{P}^{\mathrm{Ent}}$ put

$$\Psi_{\varepsilon}(\mu,\nu)=u(\mu)-v(\nu)-\frac{W(\mu,\nu)^{2}}{2\varepsilon}.$$

Then $\Psi_{\varepsilon}\le U-V$ on $\mathcal{P}^{\mathrm{Ent}}\times\mathcal{P}^{\mathrm{Ent}}$, and $\Psi_{\varepsilon}(\mu_{0},\mu_{0})=m$ by (F1), so $N_{\varepsilon}:=\sup\{\Psi_{\varepsilon}(\mu,\nu):\mu,\nu\in\mathcal{P}^{\mathrm{Ent}}\}$ is a real number with $m\le N_{\varepsilon}\le U-V$. If $0<\varepsilon\le\varepsilon'$ then $\Psi_{\varepsilon}\le\Psi_{\varepsilon'}$ pointwise, so $N_{\varepsilon}\le N_{\varepsilon'}$. Hence the sequence $(N_{2^{-k}})_{k\in\mathbb{N}}$ is nonincreasing and bounded below by $m$, and it converges to a real number $L$ by A Bounded Monotone Sequence of Real Numbers Converges §nonincreasing.

Step 3 (Order of choices). The following are chosen in this order. (i) Put $\eta=\rho m/4>0$ (recall $\rho>0$ by The Discounted Hamilton-Jacobi-Bellman Equation with Idiosyncratic Noise on the Torus Wasserstein Space: Standing Notation §parameters) and let $r>0$ be given by the structure condition for this $\eta$. (ii) Put $\theta=\tfrac12\min\{m/8,\,r/24,\,r^{2}/48\}>0$; then $2\theta<m/4$, $12\theta<r/2$ and $\sqrt{12\theta}<r/2$, so $12\theta+\sqrt{12\theta}<r$. (iii) Choose $K\in\mathbb{N}$ with $|N_{2^{-k}}-L|<\theta/2$ for all $k\ge K$, and put $\varepsilon=2^{-(K+1)}$ and $\alpha=1/\varepsilon$; then $0<\varepsilon\le1$ and $N_{2\varepsilon}-N_{\varepsilon}=N_{2^{-K}}-N_{2^{-(K+1)}}<\theta$. (iv) By the definition of the supremum choose $\mu',\nu'\in\mathcal{P}^{\mathrm{Ent}}$ with $\Psi_{\varepsilon}(\mu',\nu')>N_{\varepsilon}-\theta$. (v) Choose a real $\delta$ with $0<\delta\le\delta_{0}$, $\delta\,(\mathrm{Ent}(\mu')+\mathrm{Ent}(\nu'))<\theta$ and $2\delta\,c(\sqrt d/(2\varepsilon))<\rho m/4$; this is possible since the quantities multiplying $\delta$ are finite and nonnegative by (F5).

Step 4 (A maximiser of the penalised functional). For $\mu,\nu\in\mathcal{P}^{\mathrm{Ent}}$ put

$$\Gamma(\mu,\nu)=\Psi_{\varepsilon}(\mu,\nu)-\delta\,\bigl(\mathrm{Ent}(\mu)+\mathrm{Ent}(\nu)\bigr).$$

By (F5), $\Gamma\le\Psi_{\varepsilon}\le U-V$, so $M:=\sup\{\Gamma(\mu,\nu):\mu,\nu\in\mathcal{P}^{\mathrm{Ent}}\}$ is a real number. For each $n\in\mathbb{N}$ choose $\mu_{n},\nu_{n}\in\mathcal{P}^{\mathrm{Ent}}$ with $\Gamma(\mu_{n},\nu_{n})>M-1/(n+1)$, and put $a_{n}=\mathrm{Ent}(\mu_{n})$, $b_{n}=\mathrm{Ent}(\nu_{n})$. Then $\delta(a_{n}+b_{n})=\Psi_{\varepsilon}(\mu_{n},\nu_{n})-\Gamma(\mu_{n},\nu_{n})<U-V-M+1$, so with $C_{0}=(U-V-M+1)/\delta$ we have $0\le a_{n}\le C_{0}$ and $0\le b_{n}\le C_{0}$ by (F5).

By the Bolzano-Weierstrass theorem the bounded sequence $(a_{n})$ has a subsequence converging to a real number $a$; by The Torus Wasserstein Space is a Sequentially Compact Metric Space in which Optimal Couplings Exist §compact applied to the corresponding subsequence of $(\mu_{n})$, and then again to the resulting subsequence of $(\nu_{n})$, there are a strictly increasing sequence $(m_{j})_{j\in\mathbb{N}}$ in $\mathbb{N}$ and $\hat\mu,\hat\nu\in\mathcal{P}$ such that $(\nu_{m_{j}})$ converges to $\hat\nu$ and, by A Subsequence of a Convergent Sequence Has the Same Limit (and its elementary analogue for real sequences), $(\mu_{m_{j}})$ converges to $\hat\mu$ and $(a_{m_{j}})$ converges to $a$. By The Entropy of Measures on the Torus: Nonnegativity, Absolute Continuity and Closed Sublevel Sets §closed with $C=C_{0}$, $\hat\mu,\hat\nu\in\mathcal{P}^{\mathrm{Ent}}$.

We show $\Gamma(\hat\mu,\hat\nu)=M$. Fix $\tau>0$. By the definitions of upper and lower semicontinuity at $\hat\mu$ and $\hat\nu$ and the definition of convergence, and since $|W(\mu_{m_{j}},\nu_{m_{j}})-W(\hat\mu,\hat\nu)|\le W(\mu_{m_{j}},\hat\mu)+W(\nu_{m_{j}},\hat\nu)$ by the triangle inequality (F1), there is $J\in\mathbb{N}$ such that for every $j\ge J$: $u(\mu_{m_{j}})<u(\hat\mu)+\tau$; $v(\nu_{m_{j}})>v(\hat\nu)-\tau$; $W(\mu_{m_{j}},\nu_{m_{j}})^{2}/(2\varepsilon)>W(\hat\mu,\hat\nu)^{2}/(2\varepsilon)-\tau$; $|a_{m_{j}}-a|<\tau$; and $\Gamma(\mu_{m_{j}},\nu_{m_{j}})>M-\tau$ (as $m_{j}\ge j$). For $j\ge J$ therefore

$$\delta\,b_{m_{j}}=\Psi_{\varepsilon}(\mu_{m_{j}},\nu_{m_{j}})-\Gamma(\mu_{m_{j}},\nu_{m_{j}})-\delta\,a_{m_{j}}<\Psi_{\varepsilon}(\hat\mu,\hat\nu)-M-\delta a+(4+\delta)\tau ,$$

and $a_{m_{j}}<a+\tau$. Applying The Entropy of Measures on the Torus: Nonnegativity, Absolute Continuity and Closed Sublevel Sets §closed to the sequences $(\mu_{m_{J+j}})_{j\in\mathbb{N}}$ and $(\nu_{m_{J+j}})_{j\in\mathbb{N}}$, which converge to $\hat\mu$ and $\hat\nu$, gives $\mathrm{Ent}(\hat\mu)\le a+\tau$ and $\delta\,\mathrm{Ent}(\hat\nu)\le\Psi_{\varepsilon}(\hat\mu,\hat\nu)-M-\delta a+(4+\delta)\tau$. Hence

$$\Gamma(\hat\mu,\hat\nu)\ge\Psi_{\varepsilon}(\hat\mu,\hat\nu)-\delta(a+\tau)-\bigl(\Psi_{\varepsilon}(\hat\mu,\hat\nu)-M-\delta a+(4+\delta)\tau\bigr)=M-(4+2\delta)\tau .$$

As $\tau>0$ was arbitrary, $\Gamma(\hat\mu,\hat\nu)\ge M$; the reverse inequality holds by definition of $M$. So $\Gamma(\mu,\nu)\le\Gamma(\hat\mu,\hat\nu)$ for all $\mu,\nu\in\mathcal{P}^{\mathrm{Ent}}$.

Step 5 (Estimates at the maximiser). Write $\hat W=W(\hat\mu,\hat\nu)$. By Step 4 and choices (iv) and (v), $\Gamma(\hat\mu,\hat\nu)\ge\Gamma(\mu',\nu')=\Psi_{\varepsilon}(\mu',\nu')-\delta(\mathrm{Ent}(\mu')+\mathrm{Ent}(\nu'))>N_{\varepsilon}-2\theta$. Since $\hat\mu,\hat\nu\in\mathcal{P}^{\mathrm{Ent}}$, $\Psi_{2\varepsilon}(\hat\mu,\hat\nu)\le N_{2\varepsilon}$, and $\Psi_{\varepsilon}(\hat\mu,\hat\nu)\ge\Gamma(\hat\mu,\hat\nu)$ by (F5). Therefore

$$\frac{\hat W^{2}}{4\varepsilon}=\Psi_{2\varepsilon}(\hat\mu,\hat\nu)-\Psi_{\varepsilon}(\hat\mu,\hat\nu)\le N_{2\varepsilon}-\Gamma(\hat\mu,\hat\nu)<N_{2\varepsilon}-N_{\varepsilon}+2\theta<3\theta$$

by choice (iii). So $\alpha\hat W^{2}<12\theta$ and, as $\varepsilon\le1$, $\hat W^{2}<12\theta$; with choice (ii), $\alpha\hat W^{2}+\hat W<12\theta+\sqrt{12\theta}<r$. Moreover

$$u(\hat\mu)-v(\hat\nu)\ge\Psi_{\varepsilon}(\hat\mu,\hat\nu)\ge\Gamma(\hat\mu,\hat\nu)>N_{\varepsilon}-2\theta\ge m-2\theta .$$

By (F5), $\hat\mu,\hat\nu\in\mathcal{P}^{\mathrm{ac}}$. Fix an optimal map $T$ from $\hat\mu$ to $\hat\nu$ and an optimal map $S$ from $\hat\nu$ to $\hat\mu$, which exist by McCann's Theorem on the Flat Torus: Optimal Couplings out of an Absolutely Continuous Measure are Induced by a Unique Map with a Periodic Potential §map. By (F3) and (F1), $\lVert v_{T}\rVert_{\hat\mu}=\hat W=\lVert v_{S}\rVert_{\hat\nu}$ and $\hat W\le\sqrt d/2$.

Step 6 (The subsolution test). Put $\varphi=\alpha\Phi_{\hat\nu}+\Phi_{\hat\mu}$. By (F3) and Laplacian Test Functions on the Torus: Uniqueness of the Gradient Field, a Criterion, and Linearity of the Test Classes §linear, $\varphi$ is an intrinsic test function on $\mathcal{P}^{\mathrm{ac}}$ with $\nabla\varphi(\hat\mu)=\alpha\nabla\Phi_{\hat\nu}(\hat\mu)+\nabla\Phi_{\hat\mu}(\hat\mu)=-\alpha v_{T}$, using (F3) and (F4). For $\mu\in\mathcal{P}^{\mathrm{Ent}}$,

$$u(\mu)-\varphi(\mu)-\delta\,\mathrm{Ent}(\mu)=\Gamma(\mu,\hat\nu)+v(\hat\nu)+\delta\,\mathrm{Ent}(\hat\nu)-\tfrac12W(\mu,\hat\mu)^{2}.$$

By Step 4, $\Gamma(\mu,\hat\nu)\le\Gamma(\hat\mu,\hat\nu)$, and by (F1), $W(\mu,\hat\mu)>0=W(\hat\mu,\hat\mu)$ for $\mu\ne\hat\mu$; so $u-\varphi-\delta\,\mathrm{Ent}$ has a strict maximum over $\mathcal{P}^{\mathrm{Ent}}$ at $\hat\mu$, in the sense of Entropy-Penalised Viscosity Subsolutions, Supersolutions and Solutions of the Discounted Hamilton-Jacobi-Bellman Equation on the Torus Wasserstein Space §sub. Since $0<\delta\le\delta_{0}\le\delta_{u}$, that clause gives $\hat\mu\in\mathcal{P}^{\mathcal{I}}$ and

$$\rho\,u(\hat\mu)+H(\hat\mu,-\alpha v_{T})-\frac{\sigma^{2}\alpha}{2}\langle v_{T},\xi_{\hat\mu}\rangle_{\hat\mu}\le\delta\,c_{u}(\alpha\hat W)\le\delta\,c\bigl(\sqrt d/(2\varepsilon)\bigr),$$

where $\lVert\nabla\varphi(\hat\mu)\rVert_{\hat\mu}=\alpha\hat W\le\sqrt d/(2\varepsilon)$ by Step 5, and $c_{u}\le c$ with $c$ nondecreasing (Step 1).

Step 7 (The supersolution test). Put $\varphi'=-\alpha\Phi_{\hat\mu}-\Phi_{\hat\nu}$. By (F3), (F4) and Laplacian Test Functions on the Torus: Uniqueness of the Gradient Field, a Criterion, and Linearity of the Test Classes §linear, $\varphi'$ is an intrinsic test function on $\mathcal{P}^{\mathrm{ac}}$ with $\nabla\varphi'(\hat\nu)=-\alpha(-v_{S})-0=\alpha v_{S}$. For $\nu\in\mathcal{P}^{\mathrm{Ent}}$, using the symmetry $W(\nu,\hat\mu)=W(\hat\mu,\nu)$ of (F1),

$$v(\nu)-\varphi'(\nu)+\delta\,\mathrm{Ent}(\nu)=u(\hat\mu)-\delta\,\mathrm{Ent}(\hat\mu)-\Gamma(\hat\mu,\nu)+\tfrac12W(\nu,\hat\nu)^{2}.$$

By Step 4 and (F1) this has a strict minimum over $\mathcal{P}^{\mathrm{Ent}}$ at $\hat\nu$. Since $0<\delta\le\delta_{v}$, Entropy-Penalised Viscosity Subsolutions, Supersolutions and Solutions of the Discounted Hamilton-Jacobi-Bellman Equation on the Torus Wasserstein Space §super gives $\hat\nu\in\mathcal{P}^{\mathcal{I}}$ and, as $\lVert\nabla\varphi'(\hat\nu)\rVert_{\hat\nu}=\alpha\hat W\le\sqrt d/(2\varepsilon)$,

$$\rho\,v(\hat\nu)+H(\hat\nu,\alpha v_{S})+\frac{\sigma^{2}\alpha}{2}\langle v_{S},\xi_{\hat\nu}\rangle_{\hat\nu}\ge-\delta\,c\bigl(\sqrt d/(2\varepsilon)\bigr).$$

Step 8 (Contradiction). Subtracting the inequality of Step 7 from that of Step 6,

$$\rho\bigl(u(\hat\mu)-v(\hat\nu)\bigr)\le\bigl[H(\hat\nu,\alpha v_{S})-H(\hat\mu,-\alpha v_{T})\bigr]+\frac{\sigma^{2}\alpha}{2}\bigl[\langle\xi_{\hat\mu},v_{T}\rangle_{\hat\mu}+\langle\xi_{\hat\nu},v_{S}\rangle_{\hat\nu}\bigr]+2\delta\,c\bigl(\sqrt d/(2\varepsilon)\bigr).$$

Since $\hat\mu,\hat\nu\in\mathcal{P}^{\mathrm{Ent}}\cap\mathcal{P}^{\mathcal{I}}$ by Steps 4, 6 and 7, the second bracket is at most $0$ by McCann's Tangent Inequality on the Torus, and Monotonicity of the Score Along Optimal Maps §monotone, and $\sigma^{2}\alpha/2>0$. Since $\hat\mu,\hat\nu\in\mathcal{P}^{\mathrm{ac}}$, $\alpha>0$ and $\alpha\hat W^{2}+\hat W<r$ (Step 5), the structure condition with the $r$ of choice (i) shows that the first bracket is less than $\eta=\rho m/4$. The last term is less than $\rho m/4$ by choice (v). With Step 5,

$$\rho(m-2\theta)<\rho\bigl(u(\hat\mu)-v(\hat\nu)\bigr)<\frac{\rho m}{2},$$

so $m-2\theta<m/2$ as $\rho>0$, that is $2\theta>m/2$, contradicting $2\theta<m/4$ from choice (ii). Hence $u(\mu)\le v(\mu)$ for every $\mu\in\mathcal{P}^{\mathrm{Ent}}$.