Proof of The Gibbs Entropy Pair is a Noise Penalty Pair, with Nonnegative Penalty and Dense Score Domain

Each result cited is universally quantified over the data in its own statement.

Real order and arithmetic are those of The Real Numbers: Standing Notation and Background §background, and absolute values of real numbers obey Properties of the Absolute Value in an Ordered Field. Throughout, $\rho=\gamma_{c}$ as in A Diagonal Gaussian Reference Measure on the Noise Wasserstein Space, Rescaled Heads and Gaussian Tails: Standing Notation §gaussian, and $\mathcal{D}$, $\mathcal{D}_{\Sigma}$, $\mathcal{E}$ and $\Sigma$ are as in The Gibbs Entropy Pair on the Noise Wasserstein Space: Relative Entropy to the Gibbs Measure and the Gibbs Score Field §pair: $\mathcal{D}$ is the set of the $\mu\in\mathcal{P}(X)$ of finite relative entropy with respect to $\gamma^{V}_{\beta}$, $\mathcal{D}_{\Sigma}$ the set of the $\mu\in\mathcal{D}$ that have a relative score with respect to $\gamma^{V}_{\beta}$ and finite Fisher information relative to $\gamma^{V}_{\beta}$ with weights $a$ (The Gibbs Entropy Pair on the Noise Wasserstein Space: Relative Entropy to the Gibbs Measure and the Gibbs Score Field §penalty-domain, The Gibbs Entropy Pair on the Noise Wasserstein Space: Relative Entropy to the Gibbs Measure and the Gibbs Score Field §score-domain), $\mathcal{E}(\mu)=\beta\,H(\mu\,|\,\gamma^{V}_{\beta})$ and $\Sigma(\mu)=\beta Z^{a}_{\mu}+\nabla_{a}V$. Write $V=v\circ p_{d}$ with head dimension $d$, profile $v$ and semiconvexity constant $K\ge0$, as in Admissible Cylindrical Potentials on a Hilbert Space in the Noise Geometry §admissible, with noise gradient $\nabla_{a}V$ and functions $\partial_{k}V$ of Admissible Cylindrical Potentials on a Hilbert Space in the Noise Geometry §gradient. Fix $b\in\mathbb{R}$ as in Admissible Cylindrical Potentials on a Hilbert Space in the Noise Geometry §below and $C\in\mathbb{R}$ as in Admissible Cylindrical Potentials on a Hilbert Space in the Noise Geometry §slope; by Basic Properties of an Admissible Cylindrical Potential: Continuity, Growth under Noise Translations, Integrability, the Tangent Inequality and Tangency of the Noise Gradient §translation, $C\ge0$, and we let $C^{1/2}\ge0$ be its square root and $L=C^{1/2}(|b+1|+2)\ge0$ the number of that clause. $\exp$ obeys Basic Properties of the Exponential Function; in particular it is increasing (claim 4), positive (claim 2) and $\exp(u)^{2}=\exp(2u)$ (claim 1). For $k\in\mathbb{N}$, $a_{k}^{1/2}>0$ is the square root of $a_{k}$ and $a_{k}^{-1/2}$ its inverse, as in Lifting Euclidean Tangent Fields of the Rescaled Head to Noise Tangent Fields on a Hilbert Space. The weak form below is claim 2 of Monotonicity of Squaring on the Nonnegative Elements of an Ordered Field: for real $\alpha,\alpha'\ge0$, $\alpha\le\alpha'$ if and only if $\alpha^{2}\le\alpha'^{2}$. Integrals, their linearity and monotonicity are those of Linearity and Monotonicity of the Lebesgue Integral §nonnegative and Linearity and Monotonicity of the Lebesgue Integral §integrable; push-forwards on $X$ and $\mathbb{R}^{n}$ and the change of variables formula, including the transfer of integrability, are those of Borel Probability Measures on a Real Hilbert Space with an Orthonormal Basis: Standing Notation §pushforward and claim 2 of Image Measures, Measures with Densities, and Change of Variables.

Step 0 (pointwise bounds for the potential). Put $U(x)=V(x)+b+1$ for $x\in X$; $U$ is continuous, hence Borel, since $V$ is by Basic Properties of an Admissible Cylindrical Potential: Continuity, Growth under Noise Translations, Integrability, the Tangent Inequality and Tangency of the Noise Gradient §continuity.

(0.a) Since $-b\le V(x)$ by Basic Properties of an Admissible Cylindrical Potential: Continuity, Growth under Noise Translations, Integrability, the Tangent Inequality and Tangency of the Noise Gradient §continuity, $1\le U(x)$ for every $x\in X$.

(0.b) For $x\in X$, $|V(x)|=|U(x)-(b+1)|\le|U(x)|+|b+1|=U(x)+|b+1|$ by the triangle inequality and (0.a); as $1\le U(x)$ and $0\le1+|b+1|$, $1+|b+1|\le(1+|b+1|)U(x)$, so

$$1+|V(x)|\le(2+|b+1|)\,U(x)\qquad(x\in X).$$

(0.c) For $x\in X$, Basic Properties of an Admissible Cylindrical Potential: Continuity, Growth under Noise Translations, Integrability, the Tangent Inequality and Tangency of the Noise Gradient §continuity, the definition $\partial_{k}V(x)=\partial_{k}v(p_{d}(x))$ for $k\le d$ of Admissible Cylindrical Potentials on a Hilbert Space in the Noise Geometry §gradient, $V(x)=v(p_{d}(x))$ and Admissible Cylindrical Potentials on a Hilbert Space in the Noise Geometry §slope at $u=p_{d}(x)$ give

$$|\nabla_{a}V(x)|_{a}^{2}=\sum_{i=1}^{d}a_{i}\,\partial_{i}V(x)^{2}\le C\bigl(1+|V(x)|\bigr)^{2}.$$

Let $k\le d$. Each summand being nonnegative, $a_{k}\,\partial_{k}V(x)^{2}\le C(1+|V(x)|)^{2}\le C(2+|b+1|)^{2}U(x)^{2}$, by (0.b), the weak form and $C\ge0$. Multiplying by $a_{k}^{-1}=(a_{k}^{-1/2})^{2}$ and putting $\Gamma_{k}=a_{k}^{-1/2}C^{1/2}(2+|b+1|)\ge0$, we get $|\partial_{k}V(x)|^{2}\le(\Gamma_{k}U(x))^{2}$, and the weak form gives

$$|\partial_{k}V(x)|\le\Gamma_{k}\,U(x)\qquad(x\in X,\ k\le d).$$

(0.d) For $x\in X$ and $h\in X^{a}$, Basic Properties of an Admissible Cylindrical Potential: Continuity, Growth under Noise Translations, Integrability, the Tangent Inequality and Tangency of the Noise Gradient §translation reads $U(x+h)\le U(x)\exp(L|h|_{a})$.

Step 1 (clause 1). Let $\mu\in\mathcal{D}$. By The Gibbs Measure of an Admissible Cylindrical Potential Relative to a Diagonal Gaussian Measure §gibbs, $\gamma^{V}_{\beta}$ is a probability measure on $(X,\mathcal{B}(X))$, and so is $\mu$; $\mu$ has finite relative entropy with respect to $\gamma^{V}_{\beta}$, so the Gibbs inequality Relative Entropy on a Measurable Space: the Gibbs Inequality, the Variational Criterion and Formula, the Entropy Inequality, Small Sets and Data Processing §gibbs gives $0\le H(\mu\,|\,\gamma^{V}_{\beta})$. Since $\beta>0$, $0\le\beta\,H(\mu\,|\,\gamma^{V}_{\beta})=\mathcal{E}(\mu)$.

Step 2 (clause 2). Let $\mu\in\mathcal{D}$ and let $\varepsilon\in\mathbb{R}$ be positive. The Euclidean items cited for measures on $\mathbb{R}^{n}$ are read with $n$ in place of the dimension written $d$ (or $m$) there, as fixed in A Diagonal Gaussian Reference Measure on the Noise Wasserstein Space, Rescaled Heads and Gaussian Tails: Standing Notation §euclidean; the letter $d$ keeps its meaning as the head dimension of $V$.

(2.a) Tail replacement. By Entropy and Score Relative to the Gibbs Measure Split into Their Gaussian Parts and the Potential, with a Fisher Information Bound §entropy, $\mu$ has finite relative entropy with respect to $\gamma_{c}$, and $\mu\in\mathcal{P}_{2}(X)$ by Entropy and Score Relative to the Gibbs Measure Split into Their Gaussian Parts and the Potential, with a Fisher Information Bound §domain. By the hypothesis $c_{k}\le\kappa\,a_{k}$ and Tail Replacement in the Noise Wasserstein Distance: a Measure of Finite Relative Entropy is Approximated by the Gaussian-Tail Extensions of Its Rescaled Heads §convergence, $W_{a}(\mu,E_{m}(\tilde{\mu}_{m}))\to0$ as $m\to\infty$, so by Limit of a Sequence of Real Numbers there is $N_{0}\in\mathbb{N}$ with $W_{a}(\mu,E_{m}(\tilde{\mu}_{m}))<\varepsilon/3$ for every $m\ge N_{0}$, where $\mu,E_{m}(\tilde{\mu}_{m})\in\mathcal{P}^{a}_{\rho}$ by Tail Replacement in the Noise Wasserstein Distance: a Measure of Finite Relative Entropy is Approximated by the Gaussian-Tail Extensions of Its Rescaled Heads §membership. Let $n$ be the larger of $N_{0}$ and $d$, and put $\lambda=\tilde{\mu}_{n}=(r_{n})_{\#}\mu$; then $W_{a}(\mu,E_{n}(\lambda))<\varepsilon/3$, and $\lambda\in\mathcal{P}_{2}(\mathbb{R}^{n})$ by Lifting Euclidean Tangent Fields of the Rescaled Head to Noise Tangent Fields on a Hilbert Space §head.

(2.b) Retraction of the head onto a ball. By Radial Retraction onto a Closed Ball: Compactly Supported Approximation in the Wasserstein Distance §approximation with $m=n\ge1$, applied to $\lambda$ and $\varepsilon/3$, there is a positive $R_{0}\in\mathbb{R}$ with $W_{2}((P_{R})_{\#}\lambda,\lambda)<\varepsilon/3$ for every real $R\ge R_{0}$; put $R=R_{0}$ and $\lambda'=(P_{R})_{\#}\lambda$. Then $\lambda'\in\mathcal{P}_{2}(\mathbb{R}^{n})$ by the same clause, $W_{2}(\lambda',\lambda)<\varepsilon/3$, and $\lambda'(\bar{B}(0_{\mathbb{R}^{n}},R))=1$ by Radial Retraction onto a Closed Ball: Compactly Supported Approximation in the Wasserstein Distance §retraction.

(2.c) Gaussian smoothing. Let $s$ be the lesser of $\tfrac12$ and $\varepsilon^{2}/(18n)$, with $n$ read in $\mathbb{R}$; then $0<s\le\tfrac12$. Let $\sigma$ be the measure written $\mu_{s}$ in Gaussian Smoothing of a Measure with Finite Second Moment: Positive Density, Finite Entropy, a Tangent Score, Non-Increasing Fisher Information and Exponential Integrability, built from $\lambda'$ (in the role of $\mu$ there) and $s$. By Gaussian Smoothing of a Measure with Finite Second Moment: Positive Density, Finite Entropy, a Tangent Score, Non-Increasing Fisher Information and Exponential Integrability §density, $\sigma\in\mathcal{P}_{2}(\mathbb{R}^{n})$ and $W_{2}(\sigma,\lambda')^{2}\le n\,s\le\varepsilon^{2}/18<(\varepsilon/3)^{2}$, so $W_{2}(\sigma,\lambda')<\varepsilon/3$ by claim 1 of Monotonicity of Squaring on the Nonnegative Elements of an Ordered Field, both numbers being nonnegative. By Gaussian Smoothing of a Measure with Finite Second Moment: Positive Density, Finite Entropy, a Tangent Score, Non-Increasing Fisher Information and Exponential Integrability §entropy and Gaussian Smoothing of a Measure with Finite Second Moment: Positive Density, Finite Entropy, a Tangent Score, Non-Increasing Fisher Information and Exponential Integrability §score, $\sigma\in\mathcal{P}_{2}^{\mathrm{Ent}}(\mathbb{R}^{n})\cap\mathcal{P}_{2}^{\mathcal{I}}(\mathbb{R}^{n})$. Since $\tilde{c}^{(n)}$ is a variance vector and $\tilde{\gamma}_{n}$ the diagonal Gaussian measure with variances $\tilde{c}^{(n)}$ (A Diagonal Gaussian Reference Measure on the Noise Wasserstein Space, Rescaled Heads and Gaussian Tails: Standing Notation §heads), Relative Entropy and Relative Score with Respect to a Diagonal Gaussian Measure: Comparison with the Entropy, the Score and the Relative Free Energy of a Quadratic Potential §entropy and Relative Entropy and Relative Score with Respect to a Diagonal Gaussian Measure: Comparison with the Entropy, the Score and the Relative Free Energy of a Quadratic Potential §score, applied with the variance vector $\tilde{c}^{(n)}$ to $\sigma\in\mathcal{P}_{2}(\mathbb{R}^{n})$, show that $\sigma$ has finite relative entropy with respect to $\tilde{\gamma}_{n}$ and finite Fisher information relative to $\tilde{\gamma}_{n}$.

(2.d) The Gaussian-tail extension. Put $\nu=E_{n}(\sigma)$, the Gaussian-tail extension of $\sigma$ at level $n$. By Gaussian-Tail Extensions: the Gaussian Reference Measure, Marginals, Densities, Relative Entropy, Relative Score and the Noise Wasserstein Distance §second-moment, $\nu\in\mathcal{P}_{2}(X)$, in particular $\nu\in\mathcal{P}(X)$; by Gaussian-Tail Extensions: the Gaussian Reference Measure, Marginals, Densities, Relative Entropy, Relative Score and the Noise Wasserstein Distance §entropy, $\nu$ has finite relative entropy with respect to $\gamma_{c}$; by Gaussian-Tail Extensions: the Gaussian Reference Measure, Marginals, Densities, Relative Entropy, Relative Score and the Noise Wasserstein Distance §score and Gaussian-Tail Extensions: the Gaussian Reference Measure, Marginals, Densities, Relative Entropy, Relative Score and the Noise Wasserstein Distance §fisher, $\nu$ has a relative score with respect to $\gamma_{c}$ and finite Fisher information relative to $\gamma_{c}$ with weights $a$; and $(r_{n})_{\#}\nu=\sigma$ by Gaussian-Tail Extensions: the Gaussian Reference Measure, Marginals, Densities, Relative Entropy, Relative Score and the Noise Wasserstein Distance §head-marginal.

(2.e) The potential under $\nu$. Let $A_{0}=v(0_{\mathbb{R}^{d}})+b+1$, which satisfies $1\le A_{0}$ by Admissible Cylindrical Potentials on a Hilbert Space in the Noise Geometry §below at $u=0_{\mathbb{R}^{d}}$, and let $A=(2+|b+1|)^{2}A_{0}^{2}\ge0$. Define $h:\mathbb{R}^{n}\to\mathbb{R}$ by $h(y)=A\exp(2L\lVert y\rVert)$; then $0\le h(y)=|h(y)|$. The map $y\mapsto2L\lVert y\rVert$ is Lipschitz with constant $2L$ from $(\mathbb{R}^{n},d_{E})$ to $\mathbb{R}$ with the absolute-value metric: for $y,y'\in\mathbb{R}^{n}$, Elementary Properties of the Euclidean Norm on $\mathbb{R}^n$ §distance, Elementary Properties of the Euclidean Norm on $\mathbb{R}^n$ §homogeneity and Elementary Properties of the Euclidean Norm on $\mathbb{R}^n$ §triangle give $\lVert y\rVert\le\lVert y-y'\rVert+\lVert y'\rVert$ and $\lVert y'\rVert\le\lVert y-y'\rVert+\lVert y\rVert$, whence $|2L\lVert y\rVert-2L\lVert y'\rVert|\le2L\,d_{E}(y,y')$ by Properties of the Absolute Value in an Ordered Field §two-sided and Properties of the Absolute Value in an Ordered Field §multiplicative; so it is continuous by A Lipschitz Map is Uniformly Continuous. $\exp$ is smooth on $\mathbb{R}=\mathbb{R}^{1}$ by claim 3 of Basic Properties of the Exponential Function, hence continuous on $(\mathbb{R}^{1},d_{E})$ by claim 3 of Euclidean Space is Open in Itself, and $C^k$ Maps are Continuous; $d_{E}$ on $\mathbb{R}^{1}$ is the absolute-value metric by The Euclidean Distance on the Real Line is the Absolute Value Metric, so $\exp$ is continuous on $(\mathbb{R},d_{\mathbb{R}})$. By claim 3 of Semicontinuity and Continuity Under Composition with a Continuous Map, $y\mapsto\exp(2L\lVert y\rVert)$ is continuous, hence Borel by Probability Measures on Euclidean Space and Random Vectors: Standing Notation §borel-maps, and so is its constant multiple $h$. Since $\lambda'(\bar{B}(0_{\mathbb{R}^{n}},R))=1$ and $|h(y)|\le A\exp(2L\lVert y\rVert)$ with $A$ and $2L$ nonnegative, Gaussian Smoothing of a Measure with Finite Second Moment: Positive Density, Finite Entropy, a Tangent Score, Non-Increasing Fisher Information and Exponential Integrability §exponential shows that $h$ is integrable with respect to $\sigma$, so $\int_{\mathbb{R}^{n}}h\,d\sigma<\infty$.

Let $x\in X$. By Lifting Euclidean Tangent Fields of the Rescaled Head to Noise Tangent Fields on a Hilbert Space, $r_{n}(x)$ has coordinates $a_{k}^{-1/2}x_{k}$ ($k\le n$), so Elementary Properties of the Euclidean Norm on $\mathbb{R}^n$ §square gives $\lVert r_{n}(x)\rVert^{2}=\sum_{k=1}^{n}x_{k}^{2}/a_{k}$. Since $d\le n$ and each summand is nonnegative, $|p_{d}(x)|_{a}^{2}=\sum_{k=1}^{d}x_{k}^{2}/a_{k}\le\lVert r_{n}(x)\rVert^{2}$, and the weak form gives $|p_{d}(x)|_{a}\le\lVert r_{n}(x)\rVert$, both being nonnegative. As $L\ge0$ and $\exp$ is increasing, Basic Properties of an Admissible Cylindrical Potential: Continuity, Growth under Noise Translations, Integrability, the Tangent Inequality and Tangency of the Noise Gradient §exponential gives $U(x)\le A_{0}\exp(L|p_{d}(x)|_{a})\le A_{0}\exp(L\lVert r_{n}(x)\rVert)$. With (0.b), $1+|V(x)|\le(2+|b+1|)A_{0}\exp(L\lVert r_{n}(x)\rVert)$, both sides being nonnegative, so the weak form and $\exp(u)^{2}=\exp(2u)$ give

$$\bigl(1+|V(x)|\bigr)^{2}\le A\exp\bigl(2L\lVert r_{n}(x)\rVert\bigr)=h\bigl(r_{n}(x)\bigr)\qquad(x\in X).$$

The function $(1+|V|)^{2}$ is Borel and nonnegative, $V$ being Borel, and $h\circ r_{n}$ is Borel, $r_{n}$ being Borel by Lifting Euclidean Tangent Fields of the Rescaled Head to Noise Tangent Fields on a Hilbert Space §head. By monotonicity and the change of variables formula for the nonnegative Borel function $h$ and $(r_{n})_{\#}\nu=\sigma$,

$$\int_{X}\bigl(1+|V|\bigr)^{2}\,d\nu\le\int_{X}h\circ r_{n}\,d\nu=\int_{\mathbb{R}^{n}}h\,d\sigma<\infty .$$

Since $1\le1+|V(x)|$, $|V(x)|\le(1+|V(x)|)^{2}$, so the Borel function $V$ is integrable with respect to $\nu$. By (0.c), $|\nabla_{a}V|_{a}^{2}\le C(1+|V|)^{2}$, where $|\nabla_{a}V|_{a}^{2}=\sum_{i=1}^{d}a_{i}(\partial_{i}V)^{2}$ is Borel and nonnegative, each $\partial_{i}V$ being continuous by Basic Properties of an Admissible Cylindrical Potential: Continuity, Growth under Noise Translations, Integrability, the Tangent Inequality and Tangency of the Noise Gradient §continuity; so $\int_{X}|\nabla_{a}V|_{a}^{2}\,d\nu\le C\int_{X}(1+|V|)^{2}\,d\nu<\infty$.

(2.f) Membership. By (2.d) and (2.e), $\nu\in\mathcal{P}(X)$ has finite relative entropy with respect to $\gamma_{c}$ and $V$ is integrable with respect to $\nu$, so $\nu$ has finite relative entropy with respect to $\gamma^{V}_{\beta}$ by Entropy and Score Relative to the Gibbs Measure Split into Their Gaussian Parts and the Potential, with a Fisher Information Bound §entropy, that is, $\nu\in\mathcal{D}$. Moreover $\nu\in\mathcal{P}_{2}(X)$, $V$ is integrable with respect to $\nu$, $\nu$ has a relative score with respect to $\gamma_{c}$ and finite Fisher information relative to $\gamma_{c}$ with weights $a$, and $\int_{X}|\nabla_{a}V|_{a}^{2}\,d\nu<\infty$; by the ``if'' direction of Entropy and Score Relative to the Gibbs Measure Split into Their Gaussian Parts and the Potential, with a Fisher Information Bound §splitting, $\nu$ has a relative score with respect to $\gamma^{V}_{\beta}$ and finite Fisher information relative to $\gamma^{V}_{\beta}$ with weights $a$. Hence $\nu\in\mathcal{D}_{\Sigma}$ by The Gibbs Entropy Pair on the Noise Wasserstein Space: Relative Entropy to the Gibbs Measure and the Gibbs Score Field §score-domain.

(2.g) Distance. The measures $\sigma$, $\lambda'$ and $\lambda$ lie in $\mathcal{P}_{2}(\mathbb{R}^{n})$, so the triangle inequality The Quadratic Wasserstein Distance is a Metric on the Wasserstein Space §triangle gives $W_{2}(\sigma,\lambda)\le W_{2}(\sigma,\lambda')+W_{2}(\lambda',\lambda)<2\varepsilon/3$. By Gaussian-Tail Extensions: the Gaussian Reference Measure, Marginals, Densities, Relative Entropy, Relative Score and the Noise Wasserstein Distance §distance, applied to $\sigma,\lambda\in\mathcal{P}_{2}(\mathbb{R}^{n})$, $\nu$ and $E_{n}(\lambda)$ belong to $\mathcal{P}^{a}_{\rho}$ and $W_{a}(\nu,E_{n}(\lambda))\le W_{2}(\sigma,\lambda)<2\varepsilon/3$. By the triangle inequality The Noise Wasserstein Distance is a Metric on the Measures Noise-Connected to the Reference Measure: Existence of Noise-Optimal Couplings, Comparison with the Quadratic Wasserstein Distance and Lower Semicontinuity §triangle and the symmetry The Noise Wasserstein Distance is a Metric on the Measures Noise-Connected to the Reference Measure: Existence of Noise-Optimal Couplings, Comparison with the Quadratic Wasserstein Distance and Lower Semicontinuity §symmetry for $\nu$, $E_{n}(\lambda)$, $\mu\in\mathcal{P}^{a}_{\rho}$, and (2.a),

$$W_{a}(\nu,\mu)\le W_{a}(\nu,E_{n}(\lambda))+W_{a}(\mu,E_{n}(\lambda))<\tfrac{2\varepsilon}{3}+\tfrac{\varepsilon}{3}=\varepsilon .$$

Since $\nu\in\mathcal{D}_{\Sigma}$ by (2.f), this proves clause 2.

Step 3 (the first variation). Let $\mu\in\mathcal{D}_{\Sigma}$ and $\psi\in\mathcal{F}C^{2}_{b}(X)$. By Bounded C^2 Cylindrical Functions on a Hilbert Space with an Orthonormal Basis §cylindrical there are $n\in\mathbb{N}$ and $g\in C^{2}_{b}(\mathbb{R}^{n})$ with $\psi=g\circ p_{n}$; as recorded in Relative Entropy along a Noise-Gradient Perturbation of the Identity: the Formula, the Second-Order Expansion and the First Variation, $\psi\in\mathcal{F}C^{1}_{b}(X)$, with representation $(n,g)$. By Entropy and Score Relative to the Gibbs Measure Split into Their Gaussian Parts and the Potential, with a Fisher Information Bound §entropy, $\mu$ has finite relative entropy with respect to $\gamma_{c}$ and $V$ is integrable with respect to $\mu$; by The Gibbs Entropy Pair on the Noise Wasserstein Space: Relative Entropy to the Gibbs Measure and the Gibbs Score Field §score-domain, $\mu\in\mathcal{P}_{2}(X)$, $\mu$ has a relative score with respect to $\gamma_{c}$ and finite Fisher information relative to $\gamma_{c}$ with weights $a$, and $\int_{X}|\nabla_{a}V|_{a}^{2}\,d\mu<\infty$.

(3.a) The noise gradient of $\psi$. By The Noise Gradient of a Cylindrical Function and the Noise Tangent Space at a Probability Measure on a Hilbert Space §gradient, for $x\in X$ the $k$-th coordinate $\eta_{k}(x)$ of $\nabla_{a}\psi(x)\in X^{a}$ is $a_{k}\partial_{k}\psi(x)$ for $k\le n$ and $0$ for $k>n$, each $\eta_{k}$ is Borel, and there are real $B_{k}\ge0$ ($k\le n$) with $|\partial_{k}\psi|\le B_{k}$ on $X$ and $|\nabla_{a}\psi(x)|_{a}^{2}\le B_{\psi}$ for every $x$, where $B_{\psi}=\sum_{k=1}^{n}a_{k}B_{k}^{2}\ge0$. Put $N_{k}=a_{k}B_{k}$ for $k\le n$ and $N_{k}=0$ for $k>n$, so that $|\eta_{k}(x)|\le N_{k}$ for all $x\in X$ and $k\in\mathbb{N}$; let $b_{\psi}\ge0$ be the square root of $B_{\psi}$, so that $|\nabla_{a}\psi(x)|_{a}\le b_{\psi}$ by the weak form; and put $e_{\psi}=\exp(L\,b_{\psi})$. For $t\in\mathbb{R}$ let $T_{t}=\mathrm{id}+t\nabla_{a}\psi$, a Borel map $X\to X$ by Noise Penalty Pairs on the Noise Wasserstein Space: the Penalty, Its Score, and Their Domains (preamble), and $\mu_{t}=(T_{t})_{\#}\mu\in\mathcal{P}(X)$; $T_{0}$ is the identity, so $\mu_{0}=\mu$. For $x\in X$, $T_{t}(x)-x=t\nabla_{a}\psi(x)$ lies in the linear subspace $X^{a}$ (The Noise Space is a Real Hilbert Space: Orthonormal Basis, Continuous Embedding, Partial Sums, Closed Balls and Borel Measurability §hilbert), has coordinates $t\,\eta_{k}(x)$ and satisfies $|t\nabla_{a}\psi(x)|_{a}=|t|\,|\nabla_{a}\psi(x)|_{a}\le|t|\,b_{\psi}$; and $x-T_{t}(x)=-t\nabla_{a}\psi(x)\in X^{a}$.

(3.b) The potential part is finite. Let $t\in\mathbb{R}$ with $|t|\le1$ and $x\in X$. By (0.d) with $h=t\nabla_{a}\psi(x)$, $L\,|t\nabla_{a}\psi(x)|_{a}\le L\,b_{\psi}$ (as $L\ge0$) and monotonicity of $\exp$, with $U(x)>0$,

$$U(T_{t}(x))\le e_{\psi}\,U(x).\tag{3.1}$$

With (0.b), $|V(T_{t}(x))|\le1+|V(T_{t}(x))|\le(2+|b+1|)\,e_{\psi}\,U(x)$. The function $V\circ T_{t}$ is Borel, as a composition of Borel maps, and $U=V+b+1$ is integrable with respect to $\mu$ by linearity, $V$ being integrable and constants integrable against the probability measure $\mu$; so $V\circ T_{t}$ is integrable with respect to $\mu$ by monotonicity, and by the change of variables formula $V$ is integrable with respect to $\mu_{t}$, with

$$F(t)=\int_{X}V\,d\mu_{t}=\int_{X}V\circ T_{t}\,d\mu .$$

(3.c) The Gaussian part. By Relative Entropy along a Noise-Gradient Perturbation of the Identity: the Formula, the Second-Order Expansion and the First Variation §variation, applied to $\mu$, $n$ and $g$, there is a positive $t_{0}\in\mathbb{R}$ such that $\mu_{t}$ has finite relative entropy with respect to $\gamma_{c}$ for every $t\in(-t_{0},t_{0})$ and $f(t)=H(\mu_{t}\,|\,\gamma_{c})$ defines a function on $(-t_{0},t_{0})$ differentiable at $0$ with derivative $L^{a}_{\mu}(g)$; there, $\mathrm{id}+t\nabla_{a}\psi$ is the same map $x\mapsto x+t\nabla_{a}\psi(x)$. By The Noise Score Field of a Measure of Finite Weighted Fisher Information: Existence, Norm, Pairing with Noise Gradients, Head Approximation and Tangency §pairing-functional, $L^{a}_{\mu}(g)=\langle Z^{a}_{\mu},\nabla_{a}\psi\rangle_{\mu}$. Let $t_{1}$ be the lesser of $t_{0}$ and $1$; then $t_{1}>0$, and $0$ is an interior point of the interval $(-t_{1},t_{1})$, as recorded in Noise Penalty Pairs on the Noise Wasserstein Space: the Penalty, Its Score, and Their Domains §variation. The restriction $f_{1}$ of $f$ to $(-t_{1},t_{1})$ is differentiable at $0$ with derivative $L^{a}_{\mu}(g)$: in Derivative at an Interior Point, any $\delta$ that serves for $f$ and a given $\varepsilon$ serves for $f_{1}$, since every $h$ with $0+h\in(-t_{1},t_{1})$ has $0+h\in(-t_{0},t_{0})$ and the difference quotients agree.

(3.d) The penalty along the perturbation. Let $t\in(-t_{1},t_{1})$. Then $\mu_{t}\in\mathcal{P}(X)$ has finite relative entropy with respect to $\gamma_{c}$ by (3.c) and $V$ is integrable with respect to $\mu_{t}$ by (3.b), so by Entropy and Score Relative to the Gibbs Measure Split into Their Gaussian Parts and the Potential, with a Fisher Information Bound §entropy $\mu_{t}$ has finite relative entropy with respect to $\gamma^{V}_{\beta}$, that is, $\mu_{t}\in\mathcal{D}$, and, multiplying the formula of that clause by $\beta$,

$$\mathcal{E}(\mu_{t})=\beta\,f_{1}(t)+F(t)+\beta\log Z_{V,\beta},\tag{3.2}$$

$Z_{V,\beta}$ being a positive real number by The Gibbs Measure of an Admissible Cylindrical Potential Relative to a Diagonal Gaussian Measure §normaliser.

(3.e) Differentiability of the potential part. Put

$$G=\int_{X}\sum_{k=1}^{d}\partial_{k}V\,\eta_{k}\,d\mu .$$

This is well defined: each $\partial_{k}V$ ($k\le d$) is integrable with respect to $\mu$ by Basic Properties of an Admissible Cylindrical Potential: Continuity, Growth under Noise Translations, Integrability, the Tangent Inequality and Tangency of the Noise Gradient §integrable, and $\partial_{k}V\,\eta_{k}$ is Borel with $|\partial_{k}V\,\eta_{k}|\le N_{k}|\partial_{k}V|$. By Basic Properties of an Admissible Cylindrical Potential: Continuity, Growth under Noise Translations, Integrability, the Tangent Inequality and Tangency of the Noise Gradient §continuity with $h=\nabla_{a}\psi(x)$, $\langle\nabla_{a}V(x),\nabla_{a}\psi(x)\rangle_{a}=\sum_{k=1}^{d}\partial_{k}V(x)\,\eta_{k}(x)$. The class of $\nabla_{a}V$ lies in $L^{2}(\mu;X^{a})$ by Basic Properties of an Admissible Cylindrical Potential: Continuity, Growth under Noise Translations, Integrability, the Tangent Inequality and Tangency of the Noise Gradient §tangent, and $\nabla_{a}\psi$ is square-integrable with respect to $\mu$ by The Noise Gradient of a Cylindrical Function and the Noise Tangent Space at a Probability Measure on a Hilbert Space §gradient; so by The Space of Square-Integrable Maps from a Measure Space into a Hilbert Space with an Orthonormal Basis §operations (through Probability Measures on a Hilbert Space Transported in the Noise Norm: Standing Notation §fields),

$$G=\int_{X}\langle\nabla_{a}V,\nabla_{a}\psi\rangle_{a}\,d\mu=\langle\nabla_{a}V,\nabla_{a}\psi\rangle_{\mu}.$$

Pointwise estimate. Let $x\in X$ and $t\in\mathbb{R}$, and write $y=T_{t}(x)$. By (3.a) and Basic Properties of an Admissible Cylindrical Potential: Continuity, Growth under Noise Translations, Integrability, the Tangent Inequality and Tangency of the Noise Gradient §continuity, $\langle\nabla_{a}V(x),y-x\rangle_{a}=t\sum_{k=1}^{d}\partial_{k}V(x)\eta_{k}(x)$, $\langle\nabla_{a}V(y),x-y\rangle_{a}=-t\sum_{k=1}^{d}\partial_{k}V(y)\eta_{k}(x)$, and $|y-x|_{a}^{2}=|x-y|_{a}^{2}=t^{2}|\nabla_{a}\psi(x)|_{a}^{2}\le t^{2}B_{\psi}$. The tangent inequality Basic Properties of an Admissible Cylindrical Potential: Continuity, Growth under Noise Translations, Integrability, the Tangent Inequality and Tangency of the Noise Gradient §tangent-inequality, applied to the pair $(x,y)$ and to the pair $(y,x)$, gives with $K\ge0$

$$D_{t}(x):=V(y)-V(x)-t\sum_{k=1}^{d}\partial_{k}V(x)\eta_{k}(x)\ \ge\ -\frac{K}{2}t^{2}B_{\psi}, \qquad D_{t}(x)\ \le\ t\sum_{k=1}^{d}\bigl(\partial_{k}V(y)-\partial_{k}V(x)\bigr)\eta_{k}(x)+\frac{K}{2}t^{2}B_{\psi}.$$

Writing $\varphi_{t,k}(x)=|\partial_{k}V(T_{t}(x))-\partial_{k}V(x)|$, the triangle inequality, multiplicativity and $|\eta_{k}(x)|\le N_{k}$ bound the last sum by $|t|\sum_{k=1}^{d}N_{k}\varphi_{t,k}(x)$, and Properties of the Absolute Value in an Ordered Field §two-sided yields

$$|D_{t}(x)|\le|t|\sum_{k=1}^{d}N_{k}\,\varphi_{t,k}(x)+\frac{K}{2}t^{2}B_{\psi}\qquad(x\in X,\ t\in\mathbb{R}).\tag{3.3}$$

Integration. Let $t\in(-t_{1},t_{1})$ with $t\ne0$. By (3.b) and linearity, $D_{t}$ is integrable with respect to $\mu$ with $\int_{X}D_{t}\,d\mu=F(t)-F(0)-tG$. For $k\le d$, $\varphi_{t,k}$ is Borel, $\partial_{k}V$ being continuous by Basic Properties of an Admissible Cylindrical Potential: Continuity, Growth under Noise Translations, Integrability, the Tangent Inequality and Tangency of the Noise Gradient §continuity and $T_{t}$ Borel, and by (0.c) and (3.1) (as $|t|\le1$), $0\le\varphi_{t,k}\le|\partial_{k}V\circ T_{t}|+|\partial_{k}V|\le\Gamma_{k}(e_{\psi}+1)\,U$, an integrable function; so $\varphi_{t,k}$ is integrable. Since $-|D_{t}|\le D_{t}\le|D_{t}|$, monotonicity, (3.3) and division by $|t|>0$ give

$$\Bigl|\frac{F(t)-F(0)}{t}-G\Bigr|\le\sum_{k=1}^{d}N_{k}\int_{X}\varphi_{t,k}\,d\mu+\frac{K}{2}|t|\,B_{\psi}.\tag{3.4}$$

Limit. Let $(t_{j})_{j\in\mathbb{N}}$ be a sequence that is admissible for $F$ on $(-t_{1},t_{1})$ at $0$ in the sense of Sequential Criterion for Differentiability at an Interior Point: $t_{j}\ne0$, $t_{j}\in(-t_{1},t_{1})$, and $t_{j}\to0$. Fix $k\le d$. For $x\in X$, the distance in $X$ from $T_{t_{j}}(x)$ to $x$ is $|t_{j}|\,|\nabla_{a}\psi(x)|$, which tends to $0$ by claim 3 of Arithmetic of Limits of Real Sequences; so $T_{t_{j}}(x)\to x$ in $(X,d)$, and since $\partial_{k}V$ is continuous at $x$, Continuity Between Metric Spaces is Equivalent to Sequential Continuity §sequential gives $\partial_{k}V(T_{t_{j}}(x))\to\partial_{k}V(x)$, that is, $\varphi_{t_{j},k}(x)\to0$. The Borel functions $\varphi_{t_{j},k}$ are dominated by the integrable function $\Gamma_{k}(e_{\psi}+1)U$, so the dominated convergence theorem (its claim 3, with limit function $0$ and the measure space $(X,\mathcal{B}(X),\mu)$) gives $\int_{X}\varphi_{t_{j},k}\,d\mu\to0$. By claims 1 and 3 of Arithmetic of Limits of Real Sequences, the right side $r_{j}$ of (3.4) at $t=t_{j}$ tends to $0$. Given a real $\varepsilon>0$, choose $J$ with $r_{j}<\varepsilon$ for $j\ge J$ (Limit of a Sequence of Real Numbers); then (3.4) gives $|q_{j}-G|<\varepsilon$ for $j\ge J$, where $q_{j}=(F(t_{j})-F(0))/t_{j}$. So $q_{j}\to G$ for every admissible sequence, and claim 2 of Sequential Criterion for Differentiability at an Interior Point shows that $F$ is differentiable at $0$ on $(-t_{1},t_{1})$ with derivative $G$.

(3.f) Conclusion. By (3.2), on $(-t_{1},t_{1})$ the function $t\mapsto\mathcal{E}(\mu_{t})$ is the sum of $\beta f_{1}$, $F$ and the constant $\beta\log Z_{V,\beta}$. By claims 1 and 2 of Sum, Constant Multiple, and Product Rules for One-Dimensional Derivatives, (3.c) and (3.e), it is differentiable at $0$ with derivative

$$\beta\langle Z^{a}_{\mu},\nabla_{a}\psi\rangle_{\mu}+\langle\nabla_{a}V,\nabla_{a}\psi\rangle_{\mu}=\langle\beta Z^{a}_{\mu}+\nabla_{a}V,\nabla_{a}\psi\rangle_{\mu}=\langle\Sigma(\mu),\nabla_{a}\psi\rangle_{\mu},$$

by bilinearity of the inner product of the real Hilbert space $L^{2}(\mu;X^{a})$ (Probability Measures on a Hilbert Space Transported in the Noise Norm: Standing Notation §fields). Together with $\mu_{t}\in\mathcal{D}$ for $t\in(-t_{1},t_{1})$ from (3.d), this is the requirement of Noise Penalty Pairs on the Noise Wasserstein Space: the Penalty, Its Score, and Their Domains §variation, with $t_{1}$ in the role of $t_{0}$ there.

Step 4 (clause 3). By The Gibbs Entropy Pair on the Noise Wasserstein Space: Relative Entropy to the Gibbs Measure and the Gibbs Score Field §pair, $\mathcal{D}_{\Sigma}\subseteq\mathcal{D}\subseteq\mathcal{P}^{a}_{\rho}$, $\mathcal{E}$ is a real-valued function on $\mathcal{D}$, and $\Sigma(\mu)\in T^{a}_{\mu}$ for every $\mu\in\mathcal{D}_{\Sigma}$ by The Gibbs Entropy Pair on the Noise Wasserstein Space: Relative Entropy to the Gibbs Measure and the Gibbs Score Field §score-domain; the setting A Diagonal Gaussian Reference Measure on the Noise Wasserstein Space, Rescaled Heads and Gaussian Tails: Standing Notation §background carries that of Probability Measures on a Hilbert Space Transported in the Noise Norm: Standing Notation, with reference measure $\rho=\gamma_{c}$. We verify the four conditions of Noise Penalty Pairs on the Noise Wasserstein Space: the Penalty, Its Score, and Their Domains §pair.

Nonempty score domain (Noise Penalty Pairs on the Noise Wasserstein Space: the Penalty, Its Score, and Their Domains §nonempty). $\gamma_{c}\in\mathcal{P}(X)$ by A Diagonal Gaussian Reference Measure on the Noise Wasserstein Space, Rescaled Heads and Gaussian Tails: Standing Notation §gaussian. The constant function $1$ on $X$ is Borel and nonnegative, and $\gamma_{c}(A)=\int_{X}\mathbf{1}_{A}\cdot1\,d\gamma_{c}$ for every Borel $A$, so it is a density of $\gamma_{c}$ with respect to $\gamma_{c}$ in the sense of The Radon-Nikodym Theorem for a Finite Measure and a Sigma-Finite Measure, and Uniqueness of Densities; and $\phi(1)=1\cdot\log1=0$ for the function $\phi(s)=s\log s$ of Relative Entropy of Probability Measures §relative-entropy, since $\log1=\log(\exp(0))=0$ by claim 1 of Basic Properties of the Exponential Function and The Natural Logarithm. Hence $\phi\circ1=0$ is integrable with respect to $\gamma_{c}$ and $\gamma_{c}$ has finite relative entropy with respect to $\gamma_{c}$. By Basic Properties of an Admissible Cylindrical Potential: Continuity, Growth under Noise Translations, Integrability, the Tangent Inequality and Tangency of the Noise Gradient §gaussian, $V$ is integrable with respect to $\gamma_{c}$, so $\gamma_{c}$ has finite relative entropy with respect to $\gamma^{V}_{\beta}$ by Entropy and Score Relative to the Gibbs Measure Split into Their Gaussian Parts and the Potential, with a Fisher Information Bound §entropy, that is, $\gamma_{c}\in\mathcal{D}$. Step 2, applied to $\gamma_{c}$ and $\varepsilon=1$, gives an element of $\mathcal{D}_{\Sigma}$.

Lower bound (Noise Penalty Pairs on the Noise Wasserstein Space: the Penalty, Its Score, and Their Domains §bound). The nonnegative constant of that condition (not the slope constant $C$ fixed above) may be taken to be $0$: for $\mu\in\mathcal{D}$, $-0\cdot(1+W_{a}(\mu,\rho)^{2})=0\le\mathcal{E}(\mu)$ by Step 1.

First variation (Noise Penalty Pairs on the Noise Wasserstein Space: the Penalty, Its Score, and Their Domains §variation). This is Step 3.

Density (Noise Penalty Pairs on the Noise Wasserstein Space: the Penalty, Its Score, and Their Domains §dense). This is Step 2.

Hence $(\mathcal{D},\mathcal{D}_{\Sigma},\mathcal{E},\Sigma)$ is a noise penalty pair on $\mathcal{P}^{a}_{\rho}$, which is clause 3; clauses 1 and 2 are Steps 1 and 2.