The Gaussian Entropy Pair on the Noise Wasserstein Space: Relative Entropy and the Noise Score Field

In the setting of A Diagonal Gaussian Reference Measure on the Noise Wasserstein Space, Rescaled Heads and Gaussian Tails: Standing Notation, so that the reference measure is $\rho=\gamma_{c}$, let $\beta\in\mathbb{R}$ be positive.

1. (Hypothesis)¶ There is a positive $\kappa\in\mathbb{R}$ with $c_{k}\le\kappa\,a_{k}$ for every $k\in\mathbb{N}$.

2. (The penalty domain)¶ $\mathcal{D}$ is the set of the $\mu\in\mathcal{P}(X)$ that have finite relative entropy with respect to $\gamma_{c}$, so that $H(\mu\,|\,\gamma_{c})$ is a real number for $\mu\in\mathcal{D}$. By the hypothesis of clause 1, $\mathcal{D}\subseteq\mathcal{P}^{a}_{\rho}$ by The Entropy Domain of a Diagonal Gaussian Reference Measure Has the Noise Map Property §inclusion; and $\mathcal{D}\subseteq\mathcal{P}_{2}(X)$, the set of The Second Moment of a Borel Probability Measure on a Hilbert Space and the Probability Measures with Finite Second Moment §space, by Relative Entropy with Respect to a Diagonal Gaussian Measure on a Hilbert Space: the Moment Bound, the Cutoff Projections, and Bounded, Tight, Weakly Closed, Wasserstein-Closed and Weakly Sequentially Compact Sublevel Sets §moment.

3. (The score domain)¶ $\mathcal{D}_{\Sigma}$ is the set of the $\mu\in\mathcal{D}$ that have a relative score with respect to $\gamma_{c}$ and finite Fisher information relative to $\gamma_{c}$ with weights $a$, these notions being applicable since $\mu\in\mathcal{P}_{2}(X)$ by clause 2. For $\mu\in\mathcal{D}_{\Sigma}$, $Z^{a}_{\mu}\in L^{2}(\mu;X^{a})$ is the noise score field of $\mu$, which lies in the noise tangent space $T^{a}_{\mu}$ by The Noise Score Field of a Measure of Finite Weighted Fisher Information: Existence, Norm, Pairing with Noise Gradients, Head Approximation and Tangency §tangent; since $T^{a}_{\mu}$ is a linear subspace of $L^{2}(\mu;X^{a})$ by Linearity of the Noise Gradient, and the Noise Tangent Space is a Closed Linear Subspace §subspace, also $\beta\,Z^{a}_{\mu}\in T^{a}_{\mu}$.

4. (The Gaussian entropy pair)¶ Under the hypothesis of clause 1, the Gaussian entropy pair with temperature $\beta$ is the quadruple $(\mathcal{D},\mathcal{D}_{\Sigma},\mathcal{E},\Sigma)$ consisting of the sets $\mathcal{D}_{\Sigma}\subseteq\mathcal{D}\subseteq\mathcal{P}^{a}_{\rho}$ of clauses 2 and 3 and the functions

$$\mathcal{E}:\mathcal{D}\to\mathbb{R},\quad\mathcal{E}(\mu)=\beta\,H(\mu\,|\,\gamma_{c}),\qquad\qquad\Sigma(\mu)=\beta\,Z^{a}_{\mu}\in T^{a}_{\mu}\quad(\mu\in\mathcal{D}_{\Sigma}).$$