Proof of The First-Order Condition at a Penalised Extremum of a Noise Intrinsic Test Function on the Noise Wasserstein Space

Each result cited is universally quantified over the data in its own statement.

Throughout, the notation is that of First-Order Equations on the Noise Wasserstein Space Relative to a Noise Penalty Pair: Standing Notation; elementary order and arithmetic of real numbers, including absolute values and the least of two positive reals, is carried by The Real Numbers: Standing Notation and Background §background, in force through Probability Measures on a Hilbert Space Transported in the Noise Norm: Standing Notation §background and Borel Probability Measures on a Real Hilbert Space with an Orthonormal Basis: Standing Notation §background. For $\mu\in\mathcal{P}(X)$ the space $L^{2}(\mu;X^{a})$ is a real Hilbert space, in particular a real inner product space, whose norm $\lVert\cdot\rVert_{\mu}$ is the norm of its inner product, by The Space of Square-Integrable Hilbert-Valued Maps is a Real Hilbert Space: Coordinates and Synthesis §hilbert. The two claims are proved together: fix $s\in\{-1,1\}$ and a point $\hat{\mu}\in Q\cap\mathcal{D}_{\Sigma}$ at which the function $\chi+s\delta\mathcal{E}$ on $\mathcal{D}$, with value $\chi(\mu)+s\delta\,\mathcal{E}(\mu)$ at $\mu$, has a local maximum relative to $\mathcal{D}$ if $s=-1$, and a local minimum relative to $\mathcal{D}$ if $s=1$. For $s=-1$ this is the hypothesis of claim 1 and for $s=1$ that of claim 2. Then $\hat{\mu}\in\mathcal{D}_{\Sigma}\subseteq\mathcal{D}$ by Noise Penalty Pairs on the Noise Wasserstein Space: the Penalty, Its Score, and Their Domains §pair. By Local Maximum of a Function Relative to a Subset of a Metric Space, respectively Local Minimum of a Function Relative to a Subset of a Metric Space, fix a positive $R\in\mathbb{R}$ such that for every $\mu\in\mathcal{D}$ with $W_{a}(\hat{\mu},\mu)<R$

$$\chi(\mu)+s\delta\,\mathcal{E}(\mu)\le\chi(\hat{\mu})+s\delta\,\mathcal{E}(\hat{\mu})\quad(s=-1),\qquad\chi(\mu)+s\delta\,\mathcal{E}(\mu)\ge\chi(\hat{\mu})+s\delta\,\mathcal{E}(\hat{\mu})\quad(s=1). \tag{$\ast$}$$

We show that $\nabla\chi(\hat{\mu})=-s\delta\,\Sigma(\hat{\mu})$, which is $\nabla\chi(\hat{\mu})=\delta\,\Sigma(\hat{\mu})$ for $s=-1$ and $\nabla\chi(\hat{\mu})=-\delta\,\Sigma(\hat{\mu})$ for $s=1$.

Step 1 (the perturbed measures). Let $\psi\in\mathcal{F}C^{2}_{b}(X)$, the set of bounded $C^{2}$ cylindrical functions; then $\psi\in\mathcal{F}C^{1}_{b}(X)$ by Noise Gradients of Bounded C^2 Cylindrical Functions Are Dense in the Noise Tangent Space §inclusion. Its noise gradient $\nabla_{a}\psi:X\to X^{a}$ is measurable and square-integrable with respect to $\hat{\mu}$ by The Noise Gradient of a Cylindrical Function and the Noise Tangent Space at a Probability Measure on a Hilbert Space §gradient; its class in $L^{2}(\hat{\mu};X^{a})$ is again written $\nabla_{a}\psi$. Write $n_{\psi}=\lVert\nabla_{a}\psi\rVert_{\hat{\mu}}\ge0$. For $t\in\mathbb{R}$ let $S_{t}=\mathrm{id}+t\,\nabla_{a}\psi$ and $\nu_{t}=(S_{t})_{\#}\hat{\mu}$. By Noise Displacement and Cross Pairings Along Couplings: Bounds, Linearity, Displacement Couplings, Polarisation and a Vanishing Criterion §displacement, applied with $\hat{\mu}$ in place of its $\mu$, with $h$ the class of $\nabla_{a}\psi$ and its representative $\nabla_{a}\psi$, the map $S_{t}$ is Borel, $\pi_{t}=(\mathrm{id},S_{t})_{\#}\hat{\mu}\in\Pi^{a}(\hat{\mu},\nu_{t})$, and

$$I^{a}(\pi_{t})=t^{2}n_{\psi}^{2},\qquad\mathcal{J}^{a}(\eta,\pi_{t})=t\,\langle\eta,\nabla_{a}\psi\rangle_{\hat{\mu}}\quad\text{for every }\eta\in L^{2}(\hat{\mu};X^{a});$$

and since $\hat{\mu}\in\mathcal{D}_{\Sigma}\subseteq\mathcal{P}^{a}_{\rho}$ (Noise Penalty Pairs on the Noise Wasserstein Space: the Penalty, Its Score, and Their Domains §pair), $\nu_{t}\in\mathcal{P}^{a}_{\rho}$ by Noise Displacement and Cross Pairings Along Couplings: Bounds, Linearity, Displacement Couplings, Polarisation and a Vanishing Criterion §displacement-connected. As $|t|\,n_{\psi}\ge0$ and $(|t|\,n_{\psi})^{2}=t^{2}n_{\psi}^{2}$, the uniqueness in Existence and Uniqueness of the Nonnegative Square Root gives $\sqrt{I^{a}(\pi_{t})}=|t|\,n_{\psi}$. By The Noise Wasserstein Distance §distance, $W_{a}(\hat{\mu},\nu_{t})^{2}\le I^{a}(\pi_{t})=(|t|\,n_{\psi})^{2}$, so claim 2 of Monotonicity of Squaring on the Nonnegative Elements of an Ordered Field, both numbers being nonnegative, gives

$$W_{a}(\hat{\mu},\nu_{t})\le|t|\,n_{\psi}. \tag{1}$$

For $t=0$ the map $S_{0}$ is $\mathrm{id}$, so $\nu_{0}=\hat{\mu}$, because $\mathrm{id}^{-1}(B)=B$ for every Borel set $B$.

Step 2 (the two one-variable functions). By Noise Penalty Pairs on the Noise Wasserstein Space: the Penalty, Its Score, and Their Domains §variation, applied at $\hat{\mu}\in\mathcal{D}_{\Sigma}$ with this $\psi$, there is a positive $t_{0}\in\mathbb{R}$ such that $\nu_{t}\in\mathcal{D}$ for every $t\in(-t_{0},t_{0})$ and the function $e:(-t_{0},t_{0})\to\mathbb{R}$, $e(t)=\mathcal{E}(\nu_{t})$, is differentiable at $0$ with $e'(0)=\langle\Sigma(\hat{\mu}),\nabla_{a}\psi\rangle_{\hat{\mu}}$. The point $0$ is an interior point of $(-t_{0},t_{0})$ by Basic Facts about Intervals of the Real Line and Their Interior Points §open-interval, since $-t_{0}<0<t_{0}$.

Let $k:(-t_{0},t_{0})\to\mathbb{R}$ be $k(t)=\chi(\nu_{t})$, so that $k(0)=\chi(\hat{\mu})$. We show that $k$ is differentiable at $0$ with $k'(0)=\langle\nabla\chi(\hat{\mu}),\nabla_{a}\psi\rangle_{\hat{\mu}}$. By property (b) of Noise Intrinsic Test Functions on the Noise Wasserstein Space §test, $\chi$ is differentiable along noise couplings at $\hat{\mu}\in Q$ with gradient $\nabla\chi(\hat{\mu})$. Put $c=1+n_{\psi}$, a positive real number. Let $\varepsilon\in\mathbb{R}$ be positive, and let $\theta$ be as in Differentiability of a Function on the Noise-Connected Measures Along Noise Couplings, and Its Gradient §differentiable for the positive number $\varepsilon\,(2c)^{-1}$ in place of its $\varepsilon$. Let $t\in(-t_{0},t_{0})$ satisfy $0<|t|<\theta\,c^{-1}$. Then $\sqrt{I^{a}(\pi_{t})}=|t|\,n_{\psi}\le|t|\,c<\theta$, so $I^{a}(\pi_{t})<\theta^{2}$ by claim 1 of Monotonicity of Squaring on the Nonnegative Elements of an Ordered Field. As $\nu_{t}\in\mathcal{P}^{a}_{\rho}$ and $\pi_{t}\in\Pi^{a}(\hat{\mu},\nu_{t})$, the estimate of that clause and Step 1, with $\eta=\nabla\chi(\hat{\mu})$, give

$$\Bigl|k(t)-k(0)-t\,\bigl\langle\nabla\chi(\hat{\mu}),\nabla_{a}\psi\bigr\rangle_{\hat{\mu}}\Bigr|\le\varepsilon\,(2c)^{-1}\,|t|\,n_{\psi}\le\tfrac{\varepsilon}{2}\,|t| ,$$

the last step because $n_{\psi}\le c$. Multiplying by the positive $|t|^{-1}$ gives $\bigl|\tfrac{k(t)-k(0)}{t}-\langle\nabla\chi(\hat{\mu}),\nabla_{a}\psi\rangle_{\hat{\mu}}\bigr|\le\tfrac{\varepsilon}{2}<\varepsilon$. As $\varepsilon$ was arbitrary, with the positive number $\theta\,c^{-1}$ in the role of the radius of Derivative at an Interior Point and $0+t=t\in(-t_{0},t_{0})$, this is the differentiability of $k$ at $0$ with the stated derivative.

Step 3 (the penalised function along the curve). Let $h:(-t_{0},t_{0})\to\mathbb{R}$ be $h=k+(s\delta)e$, with value $h(t)=\chi(\nu_{t})+s\delta\,\mathcal{E}(\nu_{t})$; in particular $h(0)=\chi(\hat{\mu})+s\delta\,\mathcal{E}(\hat{\mu})$. By claim 2 of Sum, Constant Multiple, and Product Rules for One-Dimensional Derivatives, $h$ is differentiable at $0$ with

$$h'(0)=\langle\nabla\chi(\hat{\mu}),\nabla_{a}\psi\rangle_{\hat{\mu}}+s\delta\,\langle\Sigma(\hat{\mu}),\nabla_{a}\psi\rangle_{\hat{\mu}}=\bigl\langle\nabla\chi(\hat{\mu})+s\delta\,\Sigma(\hat{\mu}),\nabla_{a}\psi\bigr\rangle_{\hat{\mu}},$$

the second equality by the additivity and homogeneity of the inner product in its first argument (Real Inner Product Space §inner-product).

Let $r$ be the lesser of $t_{0}$ and $R\,c^{-1}$, a positive real number. Let $t\in(-t_{0},t_{0})$ satisfy $|0-t|<r$. Then $\nu_{t}\in\mathcal{D}$ by Step 2, and by (1), $W_{a}(\hat{\mu},\nu_{t})\le|t|\,n_{\psi}\le|t|\,c<r\,c\le R$. So $(\ast)$ applies to $\mu=\nu_{t}$ and gives $h(t)\le h(0)$ if $s=-1$ and $h(t)\ge h(0)$ if $s=1$. Since the metric of The Absolute Value Metric on the Real Line gives $d_{\mathbb{R}}(0,t)=|0-t|$, the function $h$ has a local maximum at $0$ relative to $(-t_{0},t_{0})$ if $s=-1$, and a local minimum there if $s=1$. By Vanishing of the Derivative at an Interior Local Extremum, applied with $p=-t_{0}$, $q=t_{0}$, $g=h$ and the point $0\in(-t_{0},t_{0})$, $h'(0)=0$. As $\psi$ was arbitrary,

$$\bigl\langle v,\nabla_{a}\psi\bigr\rangle_{\hat{\mu}}=0\qquad\text{for every }\psi\in\mathcal{F}C^{2}_{b}(X),\qquad\text{where }v=\nabla\chi(\hat{\mu})+s\delta\,\Sigma(\hat{\mu}). \tag{2}$$

Step 4 (vanishing in the tangent space). The fields $\nabla\chi(\hat{\mu})$ and $\Sigma(\hat{\mu})$ lie in $T^{a}_{\hat{\mu}}$, by property (b) of Noise Intrinsic Test Functions on the Noise Wasserstein Space §test and by Noise Penalty Pairs on the Noise Wasserstein Space: the Penalty, Its Score, and Their Domains §pair, and $T^{a}_{\hat{\mu}}$ is a linear subspace of $L^{2}(\hat{\mu};X^{a})$ by Linearity of the Noise Gradient, and the Noise Tangent Space is a Closed Linear Subspace §subspace; hence $v\in T^{a}_{\hat{\mu}}$ by Linear Subspace. Let $G^{a,2}_{\hat{\mu}}$ be the set of the classes in $L^{2}(\hat{\mu};X^{a})$ of the noise gradients $\nabla_{a}\psi$ of the functions $\psi\in\mathcal{F}C^{2}_{b}(X)$. By (2) and the symmetry of the inner product (Real Inner Product Space §inner-product), $\langle g,v\rangle_{\hat{\mu}}=0$ for every $g\in G^{a,2}_{\hat{\mu}}$.

Suppose, for a contradiction, that $\lVert v\rVert_{\hat{\mu}}\ne0$, so $\lVert v\rVert_{\hat{\mu}}>0$. Let $d$ be the distance $d(\xi,\zeta)=\lVert\xi-\zeta\rVert_{\hat{\mu}}$ on $L^{2}(\hat{\mu};X^{a})$ (Real Inner Product Space §distance). Since $\hat{\mu}\in\mathcal{P}^{a}_{\rho}\subseteq\mathcal{P}(X)$ and $v\in T^{a}_{\hat{\mu}}$, Noise Gradients of Bounded C^2 Cylindrical Functions Are Dense in the Noise Tangent Space §density, applied with $\hat{\mu}$ in place of its $\mu$ and with $v$, gives a sequence $(\psi_{j})_{j\in\mathbb{N}}$ in $\mathcal{F}C^{2}_{b}(X)$ with $\lVert\nabla_{a}\psi_{j}-v\rVert_{\hat{\mu}}\to0$; since $d(\nabla_{a}\psi_{j},v)=\lVert\nabla_{a}\psi_{j}-v\rVert_{\hat{\mu}}$, the classes $\nabla_{a}\psi_{j}\in G^{a,2}_{\hat{\mu}}$ converge to $v$ in $(L^{2}(\hat{\mu};X^{a}),d)$, so by Convergent Sequence in a Metric Space there is $g\in G^{a,2}_{\hat{\mu}}$ with $d(v,g)<\tfrac12\lVert v\rVert_{\hat{\mu}}$. By The Norm Metric of a Real Inner Product Space: Triangle Inequalities, Limits and Continuity §lipschitz, the map $\xi\mapsto\langle\xi,v\rangle_{\hat{\mu}}$ is Lipschitz with constant $\lVert v\rVert_{\hat{\mu}}$, so

$$\lVert v\rVert_{\hat{\mu}}^{2}=\bigl|\langle v,v\rangle_{\hat{\mu}}-\langle g,v\rangle_{\hat{\mu}}\bigr|\le\lVert v\rVert_{\hat{\mu}}\,d(v,g)<\tfrac12\lVert v\rVert_{\hat{\mu}}^{2},$$

using $\langle v,v\rangle_{\hat{\mu}}=\lVert v\rVert_{\hat{\mu}}^{2}$ (Real Inner Product Space §norm) and $\langle g,v\rangle_{\hat{\mu}}=0$. This contradicts $\lVert v\rVert_{\hat{\mu}}>0$. Hence $\lVert v\rVert_{\hat{\mu}}=0$, and $v$ is the zero vector of $L^{2}(\hat{\mu};X^{a})$ by Elementary Identities in a Real Inner Product Space §vanishing.

Step 5 (conclusion). Adding $-s\delta\,\Sigma(\hat{\mu})$ to both sides of $\nabla\chi(\hat{\mu})+s\delta\,\Sigma(\hat{\mu})=0$ in the vector space $L^{2}(\hat{\mu};X^{a})$ gives $\nabla\chi(\hat{\mu})=(-s\delta)\,\Sigma(\hat{\mu})$. For $s=-1$ this is $\nabla\chi(\hat{\mu})=\delta\,\Sigma(\hat{\mu})$, which is claim 1. For $s=1$ it is $\nabla\chi(\hat{\mu})=(-\delta)\,\Sigma(\hat{\mu})$, and $(-\delta)\,\Sigma(\hat{\mu})=(-1)\bigl(\delta\,\Sigma(\hat{\mu})\bigr)=-\bigl(\delta\,\Sigma(\hat{\mu})\bigr)$ by the scalar-multiplication axioms and claim 5 of Elementary Identities in a Vector Space; this is claim 2. $\blacksquare$