Proof of First- and Second-Order Conditions at a Penalised Extremum of an Intrinsic Test Function on the Wasserstein Space

Each result cited is universally quantified over the data in its own statement.

Claim 1. Let $\hat{\mu}\in Q\cap\mathcal{D}_{\Sigma}$ be a point at which $\chi-\delta\mathcal{E}$ has a local maximum relative to $\mathcal{D}$. By Local Maximum of a Function Relative to a Subset of a Metric Space there is a positive $R\in\mathbb{R}$ such that

$$\chi(\mu)-\delta\,\mathcal{E}(\mu)\le\chi(\hat{\mu})-\delta\,\mathcal{E}(\hat{\mu})\qquad\text{for every }\mu\in\mathcal{D}\text{ with }W_{2}(\mu,\hat{\mu})<R. \tag{$\ast$}$$

The second-order condition. Let $g:\mathbb{R}^{d}\to\mathbb{R}$ be the function $g(a)=\chi\bigl((\tau_{a})_{\#}\hat{\mu}\bigr)$; by property (d) of Intrinsic Test Functions on the Wasserstein Space and Their Translation Hessians (i) it is of class $C^{2}$ on $\mathbb{R}^{d}$, and its Hessian matrix at $0_{\mathbb{R}^{d}}$ is $H_{\chi}(\hat{\mu})$ by Intrinsic Test Functions on the Wasserstein Space and Their Translation Hessians (vii). Let $a\in\mathbb{R}^{d}$ satisfy $\lVert a\rVert<R$. By Penalty Pairs on the Wasserstein Space: the Penalty, Its Score, and Their Domains (iv) the measure $(\tau_{a})_{\#}\hat{\mu}$ belongs to $\mathcal{D}$ and $\mathcal{E}\bigl((\tau_{a})_{\#}\hat{\mu}\bigr)=\mathcal{E}(\hat{\mu})$, and $W_{2}\bigl((\tau_{a})_{\#}\hat{\mu},\hat{\mu}\bigr)\le\lVert a\rVert<R$. Indeed, the map $\tau_{a}$ is Borel, being continuous, and $\tau_{a}-\mathrm{id}$ is the constant map with value $a$, whose squared norm has integral $\lVert a\rVert^{2}$ against the probability measure $\hat{\mu}$ by The Integral of an Indicator Function is the Measure of the Set and claim 1 of Linearity and Monotonicity of the Lebesgue Integral, and the Integrability Criterion; so The Displacement Pairing of a Square-Integrable Vector Field Along a Coupling (iv), read with $S=\tau_{a}$, gives a coupling $\pi_{a}\in\Pi(\hat{\mu},(\tau_{a})_{\#}\hat{\mu})$ with $I(\pi_{a})=\lVert a\rVert^{2}$. Hence $W_{2}\bigl(\hat{\mu},(\tau_{a})_{\#}\hat{\mu}\bigr)^{2}\le\lVert a\rVert^{2}$ by The Quadratic Wasserstein Distance on Euclidean Space (i), the left-hand side equals $W_{2}\bigl((\tau_{a})_{\#}\hat{\mu},\hat{\mu}\bigr)^{2}$ by The Quadratic Wasserstein Distance is a Metric on the Wasserstein Space (i), and claim 2 of Monotonicity of Squaring on the Nonnegative Elements of an Ordered Field gives the bound, both sides being nonnegative. So $(\ast)$ applies to $\mu=(\tau_{a})_{\#}\hat{\mu}$ and reads $g(a)-\delta\,\mathcal{E}(\hat{\mu})\le g(0_{\mathbb{R}^{d}})-\delta\,\mathcal{E}(\hat{\mu})$, whence $g(a)\le g(0_{\mathbb{R}^{d}})$ by the compatibility of the order with addition, an axiom of Ordered Field. Since $d_{E}(a,0_{\mathbb{R}^{d}})=\lVert a\rVert$ by claim 2 of Properties of the Euclidean Norm on $\mathbb{R}^n$: Square, Distance, Vanishing, Coordinate Bounds, Homogeneity, Triangle Inequalities and Expansion of a Square and $\mathbb{R}^{d}$ is open in itself by claim 1 of Euclidean Space Is Open in Itself, the function $g$ has a local maximum at $0_{\mathbb{R}^{d}}$ relative to $\mathbb{R}^{d}$, so claim 1 of First- and Second-Order Conditions at a Local Extremum gives $H_{\chi}(\hat{\mu})\preceq0_{d}$.

The first-order condition. Let $\psi\in C_{c}^{\infty}(\mathbb{R}^{d})$. By Penalty Pairs on the Wasserstein Space: the Penalty, Its Score, and Their Domains (v) there is a positive $t_{0}\in\mathbb{R}$ such that $(\mathrm{id}+t\,\nabla\psi)_{\#}\hat{\mu}\in\mathcal{D}$ for every $t\in(-t_{0},t_{0})$ and such that the function $e:(-t_{0},t_{0})\to\mathbb{R}$, $e(t)=\mathcal{E}\bigl((\mathrm{id}+t\,\nabla\psi)_{\#}\hat{\mu}\bigr)$, is differentiable at $0$ with $e'(0)=\langle\Sigma(\hat{\mu}),\nabla\psi\rangle_{\hat{\mu}}$. The function $k:(-t_{0},t_{0})\to\mathbb{R}$, $k(t)=\chi\bigl((\mathrm{id}+t\,\nabla\psi)_{\#}\hat{\mu}\bigr)$, is differentiable at $0$ with $k'(0)=\langle\nabla\chi(\hat{\mu}),\nabla\psi\rangle_{\hat{\mu}}$. Indeed, for $t\in(-t_{0},t_{0})$ the map $S_{t}=\mathrm{id}+t\,\nabla\psi$ is Borel by The Push-Forward of a Probability Measure with Finite Second Moment by the Identity Perturbed along the Gradient of a Test Function: Borel, Finite Second Moment, the Diagonal Coupling and the Wasserstein Bound (i), and $S_{t}-\mathrm{id}$ is the map $x\mapsto t\,\nabla\psi(x)$, whose squared norm is integrable against $\hat{\mu}$ by The Gradient of a Test Function is Bounded and Square-Integrable, and Its Laplacian Bounded and Integrable, Against Every Probability Measure and whose class is $t\,\nabla\psi$; so The Displacement Pairing of a Square-Integrable Vector Field Along a Coupling (iv), read with $S=S_{t}$, gives a coupling $\pi_{t}\in\Pi\bigl(\hat{\mu},(S_{t})_{\#}\hat{\mu}\bigr)$ with

$$I(\pi_{t})=\lVert t\,\nabla\psi\rVert_{\hat{\mu}}^{2}=\bigl(|t|\,\lVert\nabla\psi\rVert_{\hat{\mu}}\bigr)^{2},\qquad\mathcal{J}\bigl(\nabla\chi(\hat{\mu}),\pi_{t}\bigr)=\bigl\langle\nabla\chi(\hat{\mu}),t\,\nabla\psi\bigr\rangle_{\hat{\mu}}=t\,\bigl\langle\nabla\chi(\hat{\mu}),\nabla\psi\bigr\rangle_{\hat{\mu}},$$

the norm being homogeneous and the inner product bilinear (Elementary Identities in a Real Inner Product Space (i)); thus $\sqrt{I(\pi_{t})}=|t|\,\lVert\nabla\psi\rVert_{\hat{\mu}}$ by Existence and Uniqueness of the Nonnegative Square Root, the right-hand side being nonnegative. For $t=0$ the map $S_{0}$ is $\mathrm{id}$, so $k(0)=\chi(\hat{\mu})$, and likewise $e(0)=\mathcal{E}(\hat{\mu})$. Let $\varepsilon\in\mathbb{R}$ be positive and put $c=1+\lVert\nabla\psi\rVert_{\hat{\mu}}$, a positive real number. The function $\chi$ is differentiable along couplings at $\hat{\mu}\in Q$ with gradient $\nabla\chi(\hat{\mu})$ by property (b) of Intrinsic Test Functions on the Wasserstein Space and Their Translation Hessians (i); let $\theta$ be as in Differentiability of a Function on the Wasserstein Space Along Couplings, and Its Gradient (i) for the positive number $\varepsilon\,(2c)^{-1}$ in place of its $\varepsilon$. Let $t\in(-t_{0},t_{0})$ satisfy $0<|t|<\theta\,c^{-1}$. Then $\sqrt{I(\pi_{t})}=|t|\,\lVert\nabla\psi\rVert_{\hat{\mu}}\le|t|\,c<\theta$, by claim 5 of Elementary Arithmetic in an Ordered Field applied to $\lVert\nabla\psi\rVert_{\hat{\mu}}\le c$ with the nonnegative multiplier $|t|$, and by claim 10 of Elementary Order Arithmetic in an Ordered Field applied to $|t|<\theta\,c^{-1}$ with the positive multiplier $c$; so $I(\pi_{t})<\theta^{2}$ by claim 1 of Monotonicity of Squaring on the Nonnegative Elements of an Ordered Field, and the estimate of that clause gives

$$\Bigl|k(t)-k(0)-t\,\bigl\langle\nabla\chi(\hat{\mu}),\nabla\psi\bigr\rangle_{\hat{\mu}}\Bigr|\le\varepsilon\,(2c)^{-1}\,|t|\,\lVert\nabla\psi\rVert_{\hat{\mu}}\le\tfrac{\varepsilon}{2}\,|t| ,$$

the last step because $\lVert\nabla\psi\rVert_{\hat{\mu}}\le c$. Multiplying by the positive $|t|^{-1}$, with claim 4 of Properties of the Absolute Value in an Ordered Field, gives $\bigl|\tfrac{k(t)-k(0)}{t}-\langle\nabla\chi(\hat{\mu}),\nabla\psi\rangle_{\hat{\mu}}\bigr|\le\tfrac{\varepsilon}{2}<\varepsilon$. As $\varepsilon$ was arbitrary and $0$ is an interior point of $(-t_{0},t_{0})$ (shown below), this is the differentiability of $k$ at $0$ in the sense of Derivative of a Real Function on an Interval, with the stated derivative. The point $0$ is interior to $(-t_{0},t_{0})$ by Basic Facts about Intervals of the Real Line (iv), since $-t_{0}<-0=0<t_{0}$ by claim 4 of Elementary Order Arithmetic in an Ordered Field, so by claim 2 of Derivatives of Constants, the Identity, Sums, Scalar Multiples, Products, Reciprocals, Quotients and Powers the function $h=k+(-\delta)e$, whose value at $t$ is $k(t)-\delta\,e(t)$ by claim 2 of Zero Products and Elementary Identities in a Field, is differentiable at $0$ with

$$h'(0)=\langle\nabla\chi(\hat{\mu}),\nabla\psi\rangle_{\hat{\mu}}-\delta\,\langle\Sigma(\hat{\mu}),\nabla\psi\rangle_{\hat{\mu}}=\bigl\langle\nabla\chi(\hat{\mu})-\delta\,\Sigma(\hat{\mu}),\nabla\psi\bigr\rangle_{\hat{\mu}},$$

the second equality by Elementary Identities in a Real Inner Product Space (i).

Put $c=1+\lVert\nabla\psi\rVert_{\hat{\mu}}$, positive by claim 3 of Elementary Order Arithmetic in an Ordered Field applied to $0<1$ (claim 6 of that lemma) and $0\le\lVert\nabla\psi\rVert_{\hat{\mu}}$, and let $\rho$ be the lesser of $t_{0}$ and $R\,c^{-1}$ (claim 9 of that lemma), positive because it is one of them, $c^{-1}$ being positive by claim 7 and $R\,c^{-1}$ by claim 5. For $t\in(-t_{0},t_{0})$ with $|t|<\rho$, The Push-Forward of a Probability Measure with Finite Second Moment by the Identity Perturbed along the Gradient of a Test Function: Borel, Finite Second Moment, the Diagonal Coupling and the Wasserstein Bound (iii) and claim 5 of Elementary Arithmetic in an Ordered Field with the nonnegative multiplier $|t|$, followed by claim 10 of Elementary Order Arithmetic in an Ordered Field with the positive multiplier $c$, give

$$W_{2}\bigl((\mathrm{id}+t\,\nabla\psi)_{\#}\hat{\mu},\hat{\mu}\bigr)\le|t|\,\lVert\nabla\psi\rVert_{\hat{\mu}}\le|t|\,c<\rho\,c\le R .$$

The last step holds by claim 5 of Elementary Arithmetic in an Ordered Field, applied to $\rho\le R\,c^{-1}$ with the nonnegative multiplier $c$. Hence $(\ast)$ applies to $\mu=(\mathrm{id}+t\,\nabla\psi)_{\#}\hat{\mu}$, which lies in $\mathcal{D}$, and yields $h(t)\le h(0)$ for every such $t$, since $h(0)=k(0)-\delta\,e(0)=\chi(\hat{\mu})-\delta\,\mathcal{E}(\hat{\mu})$; that is, $h$ has a local maximum at $0$ relative to $(-t_{0},t_{0})$. By Vanishing of the Derivative at a Local Extremum Strictly Inside an Interval, $h'(0)=0$, so

$$\bigl\langle\nabla\chi(\hat{\mu})-\delta\,\Sigma(\hat{\mu}),\nabla\psi\bigr\rangle_{\hat{\mu}}=0\qquad\text{for every }\psi\in C_{c}^{\infty}(\mathbb{R}^{d}).$$

Write $v=\nabla\chi(\hat{\mu})-\delta\,\Sigma(\hat{\mu})$. The field $\nabla\chi(\hat{\mu})$ lies in $T_{\hat{\mu}}$ by property (b) of Intrinsic Test Functions on the Wasserstein Space and Their Translation Hessians (i) and $\Sigma(\hat{\mu})$ lies in $T_{\hat{\mu}}$ by Penalty Pairs on the Wasserstein Space: the Penalty, Its Score, and Their Domains (i), so $v\in T_{\hat{\mu}}$, that set being a linear subspace of $L^{2}(\hat{\mu};\mathbb{R}^{d})$ by Basic Properties of the Tangent Space: Closed Subspace, the Identity Map Belongs to It, Second-Moment Limits, and Representation of Bounded Functionals on Gradients (i). The function $\ell:C_{c}^{\infty}(\mathbb{R}^{d})\to\mathbb{R}$ with constant value $0$ satisfies the hypotheses of claim 4 of Basic Properties of the Tangent Space: Closed Subspace, the Identity Map Belongs to It, Second-Moment Limits, and Representation of Bounded Functionals on Gradients with $C=0$, since $|\ell(\psi)|=0=0\cdot\lVert\nabla\psi\rVert_{\hat{\mu}}$ by claim 1 of Properties of the Absolute Value in an Ordered Field and claim 1 of Zero Products and Elementary Identities in a Field. Both $v$ and the zero element of $T_{\hat{\mu}}$ satisfy $\langle\cdot,\nabla\psi\rangle_{\hat{\mu}}=\ell(\psi)$ for every $\psi\in C_{c}^{\infty}(\mathbb{R}^{d})$, the latter by Elementary Identities in a Real Inner Product Space (ii); by the uniqueness in that claim they are equal, so $v$ is the zero element; adding $\delta\,\Sigma(\hat{\mu})$ to both sides of $\nabla\chi(\hat{\mu})+\bigl(-\delta\,\Sigma(\hat{\mu})\bigr)=0$, which is $v=0$ written out with claim 2 of Elementary Identities in a Vector Space, gives $\nabla\chi(\hat{\mu})=\delta\,\Sigma(\hat{\mu})$ by the associativity, inverse and identity axioms of the vector space $L^{2}(\hat{\mu};\mathbb{R}^{d})$.

Claim 2. Let $\hat{\mu}\in Q\cap\mathcal{D}_{\Sigma}$ be a point at which $\chi+\delta\mathcal{E}$ has a local minimum relative to $\mathcal{D}$. By Restrictions, Sums, Real Multiples and Differences of Intrinsic Test Functions on the Wasserstein Space (iii) the function $-\chi$ is an intrinsic test function on $Q$ with $\nabla(-\chi)(\nu)=-\nabla\chi(\nu)$ for every $\nu\in Q$ and $H_{-\chi}(\nu)=-H_{\chi}(\nu)$ for every $\nu\in\mathcal{P}_{2}(\mathbb{R}^{d})$. For $\mu\in\mathcal{D}$ one has $-\bigl(\chi(\mu)+\delta\,\mathcal{E}(\mu)\bigr)=(-\chi)(\mu)-\delta\,\mathcal{E}(\mu)$ by the distributivity axiom of the field $\mathbb{R}$ and claim 2 of Zero Products and Elementary Identities in a Field, so the function $(-\chi)-\delta\mathcal{E}$ on $\mathcal{D}$ is the negative of $\chi+\delta\mathcal{E}$; by claim 3 of Elementary Arithmetic in an Ordered Field, used in both directions, the inequalities defining a local minimum of $\chi+\delta\mathcal{E}$ at $\hat{\mu}$ relative to $\mathcal{D}$ are equivalent to those defining a local maximum of $(-\chi)-\delta\mathcal{E}$ there. Claim 1, applied to the intrinsic test function $-\chi$ on $Q$, therefore gives

$$-\nabla\chi(\hat{\mu})=\nabla(-\chi)(\hat{\mu})=\delta\,\Sigma(\hat{\mu}),\qquad -H_{\chi}(\hat{\mu})=H_{-\chi}(\hat{\mu})\preceq0_{d}.$$

Multiplying the first identity by $-1$ and using $(-1)\bigl((-1)w\bigr)=w$, which follows from claim 5 of Elementary Identities in a Vector Space and the scalar-multiplication axioms of the vector space $L^{2}(\hat{\mu};\mathbb{R}^{d})$, gives $\nabla\chi(\hat{\mu})=-\delta\,\Sigma(\hat{\mu})$. For the second, $-H_{\chi}(\hat{\mu})=0_{d}-H_{\chi}(\hat{\mu})$ entrywise by Difference of Real Matrices, and $0_{d}-H_{\chi}(\hat{\mu})\preceq0_{d}$ holds if and only if $0_{d}\preceq H_{\chi}(\hat{\mu})$ by Comparison with the Zero Matrix in the Positive Semidefinite Ordering.