Proof of Penalised Extrema of the Integral of a Lipschitz Semiconvex Function Relative to the Gaussian Free-Energy Pair: Finite Relative Fisher Information and the First-Order Condition

Each result cited below is universally quantified over the data in its own statement. Elementary arithmetic and order facts for real numbers (The Real Numbers: Standing Notation and Background §background) are used without citation, and linearity of integrals of integrable functions is Linearity and Monotonicity of the Lebesgue Integral §integrable. By The Gaussian Free-Energy Pair: Relative Entropy and Relative Score with Respect to a Diagonal Gaussian Measure §pair, $\mathcal{D}$ is the set of $\mu\in\mathcal{P}_{2}(\mathbb{R}^{d})$ of finite relative entropy with respect to $\gamma_{c}$, $\mathcal{D}_{\Sigma}$ is the set of $\mu\in\mathcal{D}$ of finite Fisher information relative to $\gamma_{c}$, and $\Sigma(\mu)=a\,\zeta^{c}_{\mu}\in T_{\mu}$ for $\mu\in\mathcal{D}_{\Sigma}$. We write $\ell=\ell^{c}_{\hat{\mu}}$ for the Ornstein-Uhlenbeck functional of $\hat{\mu}$.

Reduction. We treat both clauses at once, with $\sigma=1$ for clause 1 and $\sigma=-1$ for clause 2. In either case the function $\sigma w$ (value $\sigma w(x)$ at $x$) is semiconvex on $\mathbb{R}^{d}$ with constant $K$. Let $\Phi:\mathcal{D}\to\mathbb{R}$, $\Phi(\mu)=\sigma\bigl(W(\mu)-\varphi(\mu)\bigr)-\delta\,\mathcal{E}(\mu)$. For $\sigma=1$ this is the function of clause 1; for $\sigma=-1$ it is the negative of the function of clause 2, so by Local Minimum of a Function Relative to a Subset of a Metric Space and Local Maximum of a Function Relative to a Subset of a Metric Space it has a local maximum at $\hat{\mu}$ relative to $\mathcal{D}$. Thus in both cases there is a positive $r_{0}\in\mathbb{R}$ with $\Phi(\mu)\le\Phi(\hat{\mu})$ for every $\mu\in\mathcal{D}$ with $W_{2}(\hat{\mu},\mu)<r_{0}$. We prove that $\hat{\mu}\in\mathcal{D}_{\Sigma}$ and $\nabla w=\nabla\varphi(\hat{\mu})+\sigma\delta\,\Sigma(\hat{\mu})$ in $L^{2}(\hat{\mu};\mathbb{R}^{d})$, which is clause 1 for $\sigma=1$ and clause 2 for $\sigma=-1$.

Step 1 (the gradient of $w$ at $\hat{\mu}$). As $\hat{\mu}$ has finite relative entropy with respect to $\gamma_{c}$, it has finite entropy by Relative Entropy and Relative Score with Respect to a Diagonal Gaussian Measure: Comparison with the Entropy, the Score and the Relative Free Energy of a Quadratic Potential §entropy, hence is absolutely continuous by Basic Properties of the Entropy on the Wasserstein Space: Comparison with the Gaussian Relative Entropy, Lower Bound, Translation Invariance, Absolute Continuity, Closed Sublevel Sets and Lower Semicontinuity §absolutely-continuous. Let $E_{w}$ and $Dw(x)$ ($x\in E_{w}$) be as in The Almost-Everywhere Gradient of a Lipschitz Semiconvex or Semiconcave Function Is Tangent at Every Absolutely Continuous Measure. By The Almost-Everywhere Gradient of a Lipschitz Semiconvex or Semiconcave Function Is Tangent at Every Absolutely Continuous Measure §borel, $E_{w}$ is Borel, $\nabla w$ is Borel and $\lVert\nabla w(x)\rVert\le L$ for every $x$, and, as $\hat{\mu}\in\mathcal{D}\subseteq\mathcal{P}_{2}(\mathbb{R}^{d})$ is absolutely continuous, $\hat{\mu}(\mathbb{R}^{d}\setminus E_{w})=0$; by The Almost-Everywhere Gradient of a Lipschitz Semiconvex or Semiconcave Function Is Tangent at Every Absolutely Continuous Measure §semiconvex (for $\sigma=1$) or The Almost-Everywhere Gradient of a Lipschitz Semiconvex or Semiconcave Function Is Tangent at Every Absolutely Continuous Measure §semiconcave (for $\sigma=-1$), $\nabla w\in T_{\hat{\mu}}$.

Step 2 (a pointwise inequality). Let $x\in E_{w}$ and $h\in\mathbb{R}^{d}$. We show

$$\sigma\bigl(w(x+h)-w(x)-Dw(x)\cdot h\bigr)\ge-\tfrac{K}{2}\lVert h\rVert^{2}.$$

This is clear if $h=0$; let $h\ne0$. By claims 1 and 2 of A Derivative Matrix is the Jacobian Matrix, and is Unique and Gradient of a Real-Valued Function on a Euclidean Open Set, $w$ is differentiable at $x$ with a derivative matrix $J$ satisfying $Jk=Dw(x)\cdot k$ for $k\in\mathbb{R}^{d}$. Let $\varepsilon>0$; choose $r>0$ for $\varepsilon$ as in Differentiability at a Point for Maps Between Euclidean Spaces, and then $s\in(0,1]$ with $s\lVert h\rVert<r$. By Quadratic Increment Characterisation of Semiconvexity, applied to $\sigma w$ on $\mathbb{R}^{d}$ with constant $K$, the points $x+h$ and $x$ and the parameter $s$ (note $s(x+h)+(1-s)x=x+sh$),

$$\sigma\bigl(w(x+sh)-w(x)\bigr)\le s\,\sigma\bigl(w(x+h)-w(x)\bigr)+\tfrac{K}{2}s(1-s)\lVert h\rVert^{2}.$$

The point $k=sh$ satisfies $0<\lVert k\rVert=s\lVert h\rVert<r$ by Elementary Properties of the Euclidean Norm on $\mathbb{R}^n$ §homogeneity (with $\lVert h\rVert>0$ by Elementary Properties of the Euclidean Norm on $\mathbb{R}^n$ §square and Elementary Properties of the Euclidean Norm on $\mathbb{R}^n$ §vanishing, as $h\ne0$), so the choice of $r$ gives $\lVert w(x+k)-w(x)-Jk\rVert\le\varepsilon\lVert k\rVert$, the norm on the left being that of $\mathbb{R}^{1}$. Here $Jk=Dw(x)\cdot(sh)=s\,Dw(x)\cdot h$ by Bilinearity and Symmetry of the Dot Product on $\mathbb{R}^n$, and the Euclidean norm of a point of $\mathbb{R}^{1}$ is the absolute value of its coordinate by Elementary Properties of the Euclidean Norm on $\mathbb{R}^n$ §square; hence $|w(x+sh)-w(x)-s\,Dw(x)\cdot h|\le\varepsilon s\lVert h\rVert$, so, as $|\sigma|=1$, the left side is at least $s\,\sigma Dw(x)\cdot h-\varepsilon s\lVert h\rVert$. Dividing by $s>0$ and using $1-s\le1$,

$$\sigma Dw(x)\cdot h-\varepsilon\lVert h\rVert\le\sigma\bigl(w(x+h)-w(x)\bigr)+\tfrac{K}{2}\lVert h\rVert^{2}.$$

As $\varepsilon>0$ was arbitrary, the claim follows.

Step 3 (the curve). Fix $\psi\in C_{c}^{\infty}(\mathbb{R}^{d})$, let $G_{t}=\mathrm{id}+t\nabla\psi$ and $\mu_{t}=(G_{t})_{\#}\hat{\mu}$, and put $P=\lVert\nabla\psi\rVert_{\hat{\mu}}$, $A=\langle\nabla w,\nabla\psi\rangle_{\hat{\mu}}$ and $B=\langle\nabla\varphi(\hat{\mu}),\nabla\psi\rangle_{\hat{\mu}}$; here $\nabla\varphi(\hat{\mu})\in T_{\hat{\mu}}$ is the gradient along couplings of $\varphi$ at $\hat{\mu}$, which exists by Intrinsic Test Functions on the Wasserstein Space and Their Translation Hessians §differentiability and is unique by Differentiability of a Function on the Wasserstein Space Along Couplings, and Its Gradient §gradient. Let $t_{0}$ be as in The Gaussian Free-Energy Pair is a Penalty Pair: Translations, the First Variation, Lower Semicontinuity, the Moment Bound and the Map Property §variation for $\hat{\mu}$ and $\psi$, and $t_{2}$ the lesser of $t_{0}$ and $r_{0}/(P+1)$. For $t\in(-t_{2},t_{2})$: $\mu_{t}\in\mathcal{D}$ by that clause, $W_{2}(\hat{\mu},\mu_{t})\le|t|P<r_{0}$ by The Push-Forward of a Probability Measure with Finite Second Moment by the Identity Perturbed along the Gradient of a Test Function: Borel, Finite Second Moment, the Diagonal Coupling and the Wasserstein Bound §distance, and therefore $\Phi(\mu_{t})\le\Phi(\hat{\mu})$; moreover $\mu_{0}=\hat{\mu}$ by The Push-Forward of a Probability Measure with Finite Second Moment by the Identity Perturbed along the Gradient of a Test Function: Borel, Finite Second Moment, the Diagonal Coupling and the Wasserstein Bound §borel.

Step 4 ($W$ along the curve). We show that for $t\in(-t_{2},t_{2})$,

$$\sigma\bigl(W(\mu_{t})-W(\hat{\mu})\bigr)\ge t\sigma A-\tfrac{K}{2}t^{2}P^{2}.$$

The map $G_{t}$ is Borel (The Push-Forward of a Probability Measure with Finite Second Moment by the Identity Perturbed along the Gradient of a Test Function: Borel, Finite Second Moment, the Diagonal Coupling and the Wasserstein Bound §borel) and $w$ is Borel and bounded, so $w\circ G_{t}$ is Borel (Probability Measures on Euclidean Space and Random Vectors: Standing Notation §borel-maps) and bounded, and $W(\mu_{t})=\int w\circ G_{t}\,d\hat{\mu}$ by the change-of-variables formula of Probability Measures on Euclidean Space and Random Vectors: Standing Notation §pushforward. By The Gradient of a Test Function is Bounded and Square-Integrable, and Its Laplacian Bounded and Integrable, Against Every Probability Measure §gradient there is $K_{\psi}\ge0$ with $\lVert\nabla\psi(x)\rVert\le K_{\psi}$ for all $x$; by Pairs of Euclidean Points: Coordinate Projections, Pairings, the Product Measure on a Euclidean Space, Borel Norm Functions and Finite Sets §functions the function $\nabla w\cdot\nabla\psi$ is Borel with $|\nabla w\cdot\nabla\psi|\le LK_{\psi}$, and $\lVert\nabla\psi\rVert^{2}$ is Borel and bounded; their integrals against $\hat{\mu}$ are $A$ and $P^{2}$ (Square-Integrable Vector Fields Against a Probability Measure on Euclidean Space, and Test Functions: Standing Notation §l2mu). Hence

$$f_{t}=\sigma\bigl(w\circ G_{t}-w-t\,\nabla w\cdot\nabla\psi\bigr)+\tfrac{K}{2}t^{2}\lVert\nabla\psi\rVert^{2}$$

is bounded and Borel (claim 2 of Arithmetic, Absolute Values, and Pointwise Limits of Measurable Real-Valued Functions), so integrable with respect to $\hat{\mu}$ (Probability Measures on Euclidean Space and Random Vectors: Standing Notation §measures), with $\int f_{t}\,d\hat{\mu}=\sigma\bigl(W(\mu_{t})-W(\hat{\mu})-tA\bigr)+\tfrac{K}{2}t^{2}P^{2}$. For $x\in E_{w}$ one has $\nabla w(x)=Dw(x)$, and Step 2 with $h=t\nabla\psi(x)$ gives $f_{t}(x)\ge0$ (using Bilinearity and Symmetry of the Dot Product on $\mathbb{R}^n$ and Elementary Properties of the Euclidean Norm on $\mathbb{R}^n$ §homogeneity). So the negative part $f_{t}^{-}$ vanishes off the null set $\mathbb{R}^{d}\setminus E_{w}$ (Step 1), whence $\int f_{t}^{-}\,d\hat{\mu}=0$ by The Lebesgue Integral and Null Sets: Almost-Everywhere Comparison, Markov's Inequality, and Dominated Convergence Almost Everywhere §null-integral, and $\int f_{t}\,d\hat{\mu}=\int f_{t}^{+}\,d\hat{\mu}-\int f_{t}^{-}\,d\hat{\mu}\ge0$ (Integrable Function and the Lebesgue Integral). This is the claim.

Step 5 ($\varphi$ and $\mathcal{E}$ along the curve). Let $u(t)=\varphi(\mu_{t})$ and $e(t)=\mathcal{E}(\mu_{t})$ for $t\in(-t_{2},t_{2})$. Let $\varepsilon>0$ and let $\theta>0$ be given by Differentiability of a Function on the Wasserstein Space Along Couplings, and Its Gradient §differentiable at $\hat{\mu}$, for the gradient $\nabla\varphi(\hat{\mu})$ (Differentiability of a Function on the Wasserstein Space Along Couplings, and Its Gradient §gradient), with $\varepsilon/(P+1)$ in place of $\varepsilon$. For $t\in(-t_{2},t_{2})$ with $0<|t|<\theta/(P+1)$, the coupling $\pi_{t}=(\mathrm{id},G_{t})_{\#}\hat{\mu}\in\Pi(\hat{\mu},\mu_{t})$ has $I(\pi_{t})=t^{2}P^{2}<\theta^{2}$ (The Push-Forward of a Probability Measure with Finite Second Moment by the Identity Perturbed along the Gradient of a Test Function: Borel, Finite Second Moment, the Diagonal Coupling and the Wasserstein Bound §coupling), since $|t|P<\theta$, and by The Displacement Pairing of a Square-Integrable Vector Field Along a Coupling §displacement with $S=G_{t}$, whose displacement $t\nabla\psi$ is square-integrable against $\hat{\mu}$ (The Gradient of a Test Function is Bounded and Square-Integrable, and Its Laplacian Bounded and Integrable, Against Every Probability Measure §gradient), $\mathcal{J}(\nabla\varphi(\hat{\mu}),\pi_{t})=\langle\nabla\varphi(\hat{\mu}),t\nabla\psi\rangle_{\hat{\mu}}=tB$. Here, and in Steps 6 to 8, we use that the inner product of the real Hilbert space $L^{2}(\hat{\mu};\mathbb{R}^{d})$ (Square-Integrable Vector Fields Against a Probability Measure on Euclidean Space, and Test Functions: Standing Notation §l2mu) is symmetric, additive and homogeneous in each argument, by Real Inner Product Space §inner-product. Hence $|u(t)-u(0)-tB|\le\tfrac{\varepsilon}{P+1}|t|P<\varepsilon|t|$, so $u$ is differentiable at $0$ with $u'(0)=B$ (Derivative at an Interior Point). By The Gaussian Free-Energy Pair is a Penalty Pair: Translations, the First Variation, Lower Semicontinuity, the Moment Bound and the Map Property §variation and claim 2 of Restriction Stability of Continuity and of the Derivative, $e$ is differentiable at $0$ with $e'(0)=-a\,\ell(\psi)$. Finally $q(t)=-t\sigma A+\tfrac{K}{2}t^{2}P^{2}$ satisfies $q(0)=0$ and $|q(t)/t-(-\sigma A)|=\tfrac{K}{2}|t|P^{2}<\varepsilon$ whenever $0<|t|<2\varepsilon/(KP^{2}+1)$, so $q'(0)=-\sigma A$ (Derivative at an Interior Point).

Step 6 (the first variation). Let $g=\sigma u+\delta e+q$ on $(-t_{2},t_{2})$. By claim 2 of Sum, Constant Multiple, and Product Rules for One-Dimensional Derivatives, applied repeatedly, $g$ is differentiable at $0$ with $g'(0)=\sigma B-\delta a\,\ell(\psi)-\sigma A$. For $t\in(-t_{2},t_{2})$, Step 3 gives $\sigma\bigl(W(\mu_{t})-u(t)\bigr)-\delta e(t)\le\sigma\bigl(W(\hat{\mu})-u(0)\bigr)-\delta e(0)$, that is $\sigma\bigl(W(\mu_{t})-W(\hat{\mu})\bigr)\le\sigma\bigl(u(t)-u(0)\bigr)+\delta\bigl(e(t)-e(0)\bigr)$; with Step 4 this yields $g(t)\ge g(0)$. So $g$ has a local minimum at $0$ relative to $(-t_{2},t_{2})$, and $g'(0)=0$ by Vanishing of the Derivative at an Interior Local Extremum. Multiplying by $\sigma$ ($\sigma^{2}=1$),

$$\langle\nabla w-\nabla\varphi(\hat{\mu}),\nabla\psi\rangle_{\hat{\mu}}=A-B=-\sigma\delta a\,\ell(\psi)\qquad\text{for every }\psi\in C_{c}^{\infty}(\mathbb{R}^{d}).\tag{6.1}$$

Step 7 (finite Fisher information). By (6.1) and The Cauchy-Schwarz Inequality in a Real Inner Product Space in $L^{2}(\hat{\mu};\mathbb{R}^{d})$, $|\ell(\psi)|\le(\delta a)^{-1}\lVert\nabla w-\nabla\varphi(\hat{\mu})\rVert_{\hat{\mu}}\,\lVert\nabla\psi\rVert_{\hat{\mu}}$ for every $\psi$, the constant not depending on $\psi$. So $\hat{\mu}$ has finite Fisher information relative to $\gamma_{c}$ (Finite Fisher Information Relative to a Diagonal Gaussian Measure: the Ornstein-Uhlenbeck Functional, the Relative Score and the Relative Fisher Information §finite), and as $\hat{\mu}\in\mathcal{D}$, $\hat{\mu}\in\mathcal{D}_{\Sigma}$.

Step 8 (the first-order condition). By Finite Fisher Information Relative to a Diagonal Gaussian Measure: the Ornstein-Uhlenbeck Functional, the Relative Score and the Relative Fisher Information §score, $\langle\Sigma(\hat{\mu}),\nabla\psi\rangle_{\hat{\mu}}=a\langle\zeta^{c}_{\hat{\mu}},\nabla\psi\rangle_{\hat{\mu}}=-a\,\ell(\psi)$, so by (6.1) the field $\eta=\nabla w-\nabla\varphi(\hat{\mu})-\sigma\delta\,\Sigma(\hat{\mu})$ satisfies $\langle\eta,\nabla\psi\rangle_{\hat{\mu}}=0$ for every $\psi\in C_{c}^{\infty}(\mathbb{R}^{d})$. Since $T_{\hat{\mu}}$ is a linear subspace of $L^{2}(\hat{\mu};\mathbb{R}^{d})$ (Basic Properties of the Tangent Space: Closed Subspace, the Identity Map Belongs to It, Second-Moment Limits, and Representation of Bounded Functionals on Gradients §closed) containing $\nabla w$ (Step 1), $\nabla\varphi(\hat{\mu})$ (Step 3) and $\Sigma(\hat{\mu})$ (The Gaussian Free-Energy Pair: Relative Entropy and Relative Score with Respect to a Diagonal Gaussian Measure §pair), $\eta\in T_{\hat{\mu}}$. By Basic Properties of the Tangent Space: Closed Subspace, the Identity Map Belongs to It, Second-Moment Limits, and Representation of Bounded Functionals on Gradients §representation, applied to the zero functional with $C=0$, exactly one $\xi\in T_{\hat{\mu}}$ satisfies $\langle\xi,\nabla\psi\rangle_{\hat{\mu}}=0$ for every $\psi$; both $\eta$ and $0$ do, so $\eta=0$, that is $\nabla w=\nabla\varphi(\hat{\mu})+\sigma\delta\,\Sigma(\hat{\mu})$. $\blacksquare$