Proof of First- and Second-Order Conditions at a Penalised Extremum of an Intrinsic Test Function on the Wasserstein Space

Each result cited is universally quantified over the data in its own statement.

Claim 1. Let $\hat{\mu}\in Q\cap\mathcal{D}_{\Sigma}$ be a point at which $\chi-\delta\mathcal{E}$ has a local maximum relative to $\mathcal{D}$. By Local Maximum of a Function Relative to a Subset of a Metric Space there is a positive $R\in\mathbb{R}$ such that

The second-order condition. Let $g:\mathbb{R}^{d}\to\mathbb{R}$ be the function $g(a)=\chi\bigl((\tau_{a})_{\#}\hat{\mu}\bigr)$; by property (d) of Intrinsic Test Functions on the Wasserstein Space and Their Translation Hessians §test it is of class $C^{2}$ on $\mathbb{R}^{d}$, and its Hessian matrix at $0_{\mathbb{R}^{d}}$ is $H_{\chi}(\hat{\mu})$ by Intrinsic Test Functions on the Wasserstein Space and Their Translation Hessians §hessian. By Penalty Pairs on the Wasserstein Space: the Penalty, Its Score, and Their Domains §translation the function $e_{\hat{\mu}}:\mathbb{R}^{d}\to\mathbb{R}$, $e_{\hat{\mu}}(a)=\mathcal{E}\bigl((\tau_{a})_{\#}\hat{\mu}\bigr)$, is of class $C^{2}$ on $\mathbb{R}^{d}$, and its Hessian matrix at $0_{\mathbb{R}^{d}}$ is $H_{\mathcal{E}}(\hat{\mu})$ by Penalty Pairs on the Wasserstein Space: the Penalty, Its Score, and Their Domains §hessian. Let $w=g+(-\delta)e_{\hat{\mu}}$, whose value at $a$ is $g(a)-\delta\,e_{\hat{\mu}}(a)$ by claim 2 of Zero Products and Elementary Identities in a Field; it is of class $C^{2}$ on $\mathbb{R}^{d}$ by claim 3 of Constants, Coordinate Functions, Sums and Products of $C^k$ Functions on a Euclidean Open Set. Claim 1 of that lemma, applied at every point of $\mathbb{R}^{d}$ to the first partial derivatives of $g$ and $e_{\hat{\mu}}$, which exist everywhere, gives $\partial_{i}w=\partial_{i}g+(-\delta)\,\partial_{i}e_{\hat{\mu}}$ on $\mathbb{R}^{d}$ for every $i\in[d]$; applied again at $0_{\mathbb{R}^{d}}$ to these functions, whose partial derivatives exist there because $g$ and $e_{\hat{\mu}}$ are of class $C^{2}$, it gives $\partial_{j}\partial_{i}w(0_{\mathbb{R}^{d}})=\partial_{j}\partial_{i}g(0_{\mathbb{R}^{d}})+(-\delta)\,\partial_{j}\partial_{i}e_{\hat{\mu}}(0_{\mathbb{R}^{d}})$ for all $i,j\in[d]$; by Hessian Matrix of a C^2 Function, Difference of Real Matrices and the entrywise real multiple of Real Matrices, Symmetric Matrices and the Semidefinite Ordering: Standing Notation §matrices, this says $D^{2}w(0_{\mathbb{R}^{d}})=H_{\chi}(\hat{\mu})-\delta H_{\mathcal{E}}(\hat{\mu})$. Let $a\in\mathbb{R}^{d}$ satisfy $\lVert a\rVert<R$. By Penalty Pairs on the Wasserstein Space: the Penalty, Its Score, and Their Domains §translation the measure $(\tau_{a})_{\#}\hat{\mu}$ belongs to $\mathcal{D}$, and $W_{2}\bigl((\tau_{a})_{\#}\hat{\mu},\hat{\mu}\bigr)\le\lVert a\rVert<R$. Indeed, the map $\tau_{a}$ is Borel, being continuous, and $\tau_{a}-\mathrm{id}$ is the constant map with value $a$, whose squared norm has integral $\lVert a\rVert^{2}$ against the probability measure $\hat{\mu}$ by The Integral of an Indicator Function is the Measure of the Set and claim 1 of Linearity and Monotonicity of the Lebesgue Integral; so The Displacement Pairing of a Square-Integrable Vector Field Along a Coupling §displacement, read with $S=\tau_{a}$, gives a coupling $\pi_{a}\in\Pi(\hat{\mu},(\tau_{a})_{\#}\hat{\mu})$ with $I(\pi_{a})=\lVert a\rVert^{2}$. Hence $W_{2}\bigl(\hat{\mu},(\tau_{a})_{\#}\hat{\mu}\bigr)^{2}\le\lVert a\rVert^{2}$ by The Quadratic Wasserstein Distance on Euclidean Space §distance, the left-hand side equals $W_{2}\bigl((\tau_{a})_{\#}\hat{\mu},\hat{\mu}\bigr)^{2}$ by The Quadratic Wasserstein Distance is a Metric on the Wasserstein Space §symmetry, and claim 2 of Monotonicity of Squaring on the Nonnegative Elements of an Ordered Field gives the bound, both sides being nonnegative. So $(\ast)$ applies to $\mu=(\tau_{a})_{\#}\hat{\mu}$ and reads $w(a)\le w(0_{\mathbb{R}^{d}})$, since $(\tau_{0_{\mathbb{R}^{d}}})_{\#}\hat{\mu}=\hat{\mu}$, the translation by the origin being the identity map. Since $d_{E}(a,0_{\mathbb{R}^{d}})=\lVert a\rVert$ by claim 2 of Elementary Properties of the Euclidean Norm on $\mathbb{R}^n$ and $\mathbb{R}^{d}$ is open in itself by claim 1 of Euclidean Space is Open in Itself, and $C^k$ Maps are Continuous, the function $w$ has a local maximum at $0_{\mathbb{R}^{d}}$ relative to $\mathbb{R}^{d}$, so claim 1 of First- and Second-Order Conditions at a Local Extremum of a Function of Class $C^2$ gives $H_{\chi}(\hat{\mu})-\delta H_{\mathcal{E}}(\hat{\mu})=D^{2}w(0_{\mathbb{R}^{d}})\preceq0_{d}$, where $0_{d}$ is the zero matrix, and claim 1 of Comparison with the Zero Matrix in the Positive Semidefinite Ordering gives $H_{\chi}(\hat{\mu})\preceq\delta H_{\mathcal{E}}(\hat{\mu})$.

The first-order condition. Let $\psi\in C_{c}^{\infty}(\mathbb{R}^{d})$. By Penalty Pairs on the Wasserstein Space: the Penalty, Its Score, and Their Domains §variation there is a positive $t_{0}\in\mathbb{R}$ such that $(\mathrm{id}+t\,\nabla\psi)_{\#}\hat{\mu}\in\mathcal{D}$ for every $t\in(-t_{0},t_{0})$ and such that the function $e:(-t_{0},t_{0})\to\mathbb{R}$, $e(t)=\mathcal{E}\bigl((\mathrm{id}+t\,\nabla\psi)_{\#}\hat{\mu}\bigr)$, is differentiable at $0$ with $e'(0)=\langle\Sigma(\hat{\mu}),\nabla\psi\rangle_{\hat{\mu}}$. The function $k:(-t_{0},t_{0})\to\mathbb{R}$, $k(t)=\chi\bigl((\mathrm{id}+t\,\nabla\psi)_{\#}\hat{\mu}\bigr)$, is differentiable at $0$ with $k'(0)=\langle\nabla\chi(\hat{\mu}),\nabla\psi\rangle_{\hat{\mu}}$. Indeed, for $t\in(-t_{0},t_{0})$ the map $S_{t}=\mathrm{id}+t\,\nabla\psi$ is Borel by The Push-Forward of a Probability Measure with Finite Second Moment by the Identity Perturbed along the Gradient of a Test Function: Borel, Finite Second Moment, the Diagonal Coupling and the Wasserstein Bound §borel, and $S_{t}-\mathrm{id}$ is the map $x\mapsto t\,\nabla\psi(x)$, whose squared norm is integrable against $\hat{\mu}$ by The Gradient of a Test Function is Bounded and Square-Integrable, and Its Laplacian Bounded and Integrable, Against Every Probability Measure and whose class is $t\,\nabla\psi$; so The Displacement Pairing of a Square-Integrable Vector Field Along a Coupling §displacement, read with $S=S_{t}$, gives a coupling $\pi_{t}\in\Pi\bigl(\hat{\mu},(S_{t})_{\#}\hat{\mu}\bigr)$ with

the norm being homogeneous and the inner product bilinear (Elementary Identities in a Real Inner Product Space §bilinear); thus $\sqrt{I(\pi_{t})}=|t|\,\lVert\nabla\psi\rVert_{\hat{\mu}}$ by Existence and Uniqueness of the Nonnegative Square Root, the right-hand side being nonnegative. For $t=0$ the map $S_{0}$ is $\mathrm{id}$, so $k(0)=\chi(\hat{\mu})$, and likewise $e(0)=\mathcal{E}(\hat{\mu})$. Let $\varepsilon\in\mathbb{R}$ be positive and put $c=1+\lVert\nabla\psi\rVert_{\hat{\mu}}$, a positive real number. The function $\chi$ is differentiable along couplings at $\hat{\mu}\in Q$ with gradient $\nabla\chi(\hat{\mu})$ by property (b) of Intrinsic Test Functions on the Wasserstein Space and Their Translation Hessians §test; let $\theta$ be as in Differentiability of a Function on the Wasserstein Space Along Couplings, and Its Gradient §differentiable for the positive number $\varepsilon\,(2c)^{-1}$ in place of its $\varepsilon$. Let $t\in(-t_{0},t_{0})$ satisfy $0<|t|<\theta\,c^{-1}$. Then $\sqrt{I(\pi_{t})}=|t|\,\lVert\nabla\psi\rVert_{\hat{\mu}}\le|t|\,c<\theta$, by claim 5 of Elementary Arithmetic in an Ordered Field applied to $\lVert\nabla\psi\rVert_{\hat{\mu}}\le c$ with the nonnegative multiplier $|t|$, and by claim 10 of Elementary Order Arithmetic in an Ordered Field applied to $|t|<\theta\,c^{-1}$ with the positive multiplier $c$; so $I(\pi_{t})<\theta^{2}$ by claim 1 of Monotonicity of Squaring on the Nonnegative Elements of an Ordered Field, and the estimate of that clause gives

the last step because $\lVert\nabla\psi\rVert_{\hat{\mu}}\le c$. Multiplying by the positive $|t|^{-1}$, with claim 4 of Properties of the Absolute Value in an Ordered Field, gives $\bigl|\tfrac{k(t)-k(0)}{t}-\langle\nabla\chi(\hat{\mu}),\nabla\psi\rangle_{\hat{\mu}}\bigr|\le\tfrac{\varepsilon}{2}<\varepsilon$. As $\varepsilon$ was arbitrary and $0$ is an interior point of $(-t_{0},t_{0})$ (shown below), this is the differentiability of $k$ at $0$ in the sense of Derivative at an Interior Point, with the stated derivative. The point $0$ is interior to $(-t_{0},t_{0})$ by Basic Facts about Intervals of the Real Line and Their Interior Points §open-interval, since $-t_{0}<-0=0<t_{0}$ by claim 4 of Elementary Order Arithmetic in an Ordered Field, so by claim 2 of Sum, Constant Multiple, and Product Rules for One-Dimensional Derivatives the function $h=k+(-\delta)e$, whose value at $t$ is $k(t)-\delta\,e(t)$ by claim 2 of Zero Products and Elementary Identities in a Field, is differentiable at $0$ with

the second equality by Elementary Identities in a Real Inner Product Space §bilinear.

Put $c=1+\lVert\nabla\psi\rVert_{\hat{\mu}}$, positive by claim 3 of Elementary Order Arithmetic in an Ordered Field applied to $0<1$ (claim 6 of that lemma) and $0\le\lVert\nabla\psi\rVert_{\hat{\mu}}$, and let $\rho$ be the lesser of $t_{0}$ and $R\,c^{-1}$ (claim 9 of that lemma), positive because it is one of them, $c^{-1}$ being positive by claim 7 and $R\,c^{-1}$ by claim 5. For $t\in(-t_{0},t_{0})$ with $|t|<\rho$, The Push-Forward of a Probability Measure with Finite Second Moment by the Identity Perturbed along the Gradient of a Test Function: Borel, Finite Second Moment, the Diagonal Coupling and the Wasserstein Bound §distance and claim 5 of Elementary Arithmetic in an Ordered Field with the nonnegative multiplier $|t|$, followed by claim 10 of Elementary Order Arithmetic in an Ordered Field with the positive multiplier $c$, give

The last step holds by claim 5 of Elementary Arithmetic in an Ordered Field, applied to $\rho\le R\,c^{-1}$ with the nonnegative multiplier $c$. Hence $(\ast)$ applies to $\mu=(\mathrm{id}+t\,\nabla\psi)_{\#}\hat{\mu}$, which lies in $\mathcal{D}$, and yields $h(t)\le h(0)$ for every such $t$, since $h(0)=k(0)-\delta\,e(0)=\chi(\hat{\mu})-\delta\,\mathcal{E}(\hat{\mu})$; that is, $h$ has a local maximum at $0$ relative to $(-t_{0},t_{0})$. By Vanishing of the Derivative at an Interior Local Extremum, $h'(0)=0$, so

Write $v=\nabla\chi(\hat{\mu})-\delta\,\Sigma(\hat{\mu})$. The field $\nabla\chi(\hat{\mu})$ lies in $T_{\hat{\mu}}$ by property (b) of Intrinsic Test Functions on the Wasserstein Space and Their Translation Hessians §test and $\Sigma(\hat{\mu})$ lies in $T_{\hat{\mu}}$ by Penalty Pairs on the Wasserstein Space: the Penalty, Its Score, and Their Domains §pair, so $v\in T_{\hat{\mu}}$, that set being a linear subspace of $L^{2}(\hat{\mu};\mathbb{R}^{d})$ by Basic Properties of the Tangent Space: Closed Subspace, the Identity Map Belongs to It, Second-Moment Limits, and Representation of Bounded Functionals on Gradients §closed. The function $\ell:C_{c}^{\infty}(\mathbb{R}^{d})\to\mathbb{R}$ with constant value $0$ satisfies the hypotheses of claim 4 of Basic Properties of the Tangent Space: Closed Subspace, the Identity Map Belongs to It, Second-Moment Limits, and Representation of Bounded Functionals on Gradients with $C=0$, since $|\ell(\psi)|=0=0\cdot\lVert\nabla\psi\rVert_{\hat{\mu}}$ by claim 1 of Properties of the Absolute Value in an Ordered Field and claim 1 of Zero Products and Elementary Identities in a Field. Both $v$ and the zero element of $T_{\hat{\mu}}$ satisfy $\langle\cdot,\nabla\psi\rangle_{\hat{\mu}}=\ell(\psi)$ for every $\psi\in C_{c}^{\infty}(\mathbb{R}^{d})$, the latter by Elementary Identities in a Real Inner Product Space §zero; by the uniqueness in that claim they are equal, so $v$ is the zero element; adding $\delta\,\Sigma(\hat{\mu})$ to both sides of $\nabla\chi(\hat{\mu})+\bigl(-\delta\,\Sigma(\hat{\mu})\bigr)=0$, which is $v=0$ written out with claim 2 of Elementary Identities in a Vector Space, gives $\nabla\chi(\hat{\mu})=\delta\,\Sigma(\hat{\mu})$ by the associativity, inverse and identity axioms of the vector space $L^{2}(\hat{\mu};\mathbb{R}^{d})$.

Claim 2. Let $\hat{\mu}\in Q\cap\mathcal{D}_{\Sigma}$ be a point at which $\chi+\delta\mathcal{E}$ has a local minimum relative to $\mathcal{D}$. By Restrictions, Sums, Real Multiples and Differences of Intrinsic Test Functions on the Wasserstein Space §difference the function $-\chi$ is an intrinsic test function on $Q$ with $\nabla(-\chi)(\nu)=-\nabla\chi(\nu)$ for every $\nu\in Q$ and $H_{-\chi}(\nu)=-H_{\chi}(\nu)$ for every $\nu\in\mathcal{P}_{2}(\mathbb{R}^{d})$. For $\mu\in\mathcal{D}$ one has $-\bigl(\chi(\mu)+\delta\,\mathcal{E}(\mu)\bigr)=(-\chi)(\mu)-\delta\,\mathcal{E}(\mu)$ by the distributivity axiom of the field $\mathbb{R}$ and claim 2 of Zero Products and Elementary Identities in a Field, so the function $(-\chi)-\delta\mathcal{E}$ on $\mathcal{D}$ is the negative of $\chi+\delta\mathcal{E}$; by claim 3 of Elementary Arithmetic in an Ordered Field, used in both directions, the inequalities defining a local minimum of $\chi+\delta\mathcal{E}$ at $\hat{\mu}$ relative to $\mathcal{D}$ are equivalent to those defining a local maximum of $(-\chi)-\delta\mathcal{E}$ there. Claim 1, applied to the intrinsic test function $-\chi$ on $Q$, therefore gives

Multiplying the first identity by $-1$ and using $(-1)\bigl((-1)\zeta\bigr)=\zeta$ for $\zeta\in L^{2}(\hat{\mu};\mathbb{R}^{d})$, which follows from claim 5 of Elementary Identities in a Vector Space and the scalar-multiplication axioms of the vector space $L^{2}(\hat{\mu};\mathbb{R}^{d})$, gives $\nabla\chi(\hat{\mu})=-\delta\,\Sigma(\hat{\mu})$. For the second, write $h_{ij}$ and $c_{ij}$ for the entries of $H_{\mathcal{E}}(\hat{\mu})$ and $H_{\chi}(\hat{\mu})$. By Difference of Real Matrices and the entrywise real multiples of Real Matrices, Symmetric Matrices and the Semidefinite Ordering: Standing Notation §matrices, the difference $\delta H_{\mathcal{E}}(\hat{\mu})-\bigl(-H_{\chi}(\hat{\mu})\bigr)$ has entries $\delta h_{ij}-(-c_{ij})$ and the difference $H_{\chi}(\hat{\mu})-\bigl(-\delta H_{\mathcal{E}}(\hat{\mu})\bigr)$ has entries $c_{ij}-(-\delta h_{ij})$; both equal $c_{ij}+\delta h_{ij}$ by claim 2 of Zero Products and Elementary Identities in a Field and the commutativity of addition, so the two differences are the same matrix. By The Positive Semidefinite Ordering on Symmetric Matrices, $-H_{\chi}(\hat{\mu})\preceq\delta H_{\mathcal{E}}(\hat{\mu})$ says that the first is positive semidefinite, and $-\delta H_{\mathcal{E}}(\hat{\mu})\preceq H_{\chi}(\hat{\mu})$ says that the second is; hence $-\delta H_{\mathcal{E}}(\hat{\mu})\preceq H_{\chi}(\hat{\mu})$.