Gaussian-Tail Extensions: the Gaussian Reference Measure, Marginals, Densities, Relative Entropy, Relative Score and the Noise Wasserstein Distance

In the setting of A Diagonal Gaussian Reference Measure on the Noise Wasserstein Space, Rescaled Heads and Gaussian Tails: Standing Notation, let $n\in\mathbb{N}$; for $\lambda\in\mathcal{P}(\mathbb{R}^{n})$, $E_{n}(\lambda)$ is its Gaussian-tail extension at level $n$. Densities are those of The Radon-Nikodym Theorem for a Finite Measure and a Sigma-Finite Measure, and Uniqueness of Densities.

1. (The rescaled head of the Gaussian measure)¶ $(r_{n})_{\#}\gamma_{c}=\tilde{\gamma}_{n}$, and $\tilde{\gamma}_{n}\in\mathcal{P}_{2}(\mathbb{R}^{n})$.

2. (Reconstruction)¶ $\gamma_{c}=E_{n}(\tilde{\gamma}_{n})$.

3. (Head marginal)¶ For $\lambda\in\mathcal{P}(\mathbb{R}^{n})$, $(r_{n})_{\#}E_{n}(\lambda)=\lambda$.

4. (Tail marginal)¶ For $\lambda\in\mathcal{P}(\mathbb{R}^{n})$, $(Q_{n})_{\#}E_{n}(\lambda)=\tau_{n}$.

5. (Second moment)¶ If $\lambda\in\mathcal{P}_{2}(\mathbb{R}^{n})$, then $E_{n}(\lambda)\in\mathcal{P}_{2}(X)$, the set of The Second Moment of a Borel Probability Measure on a Hilbert Space and the Probability Measures with Finite Second Moment §space.

6. (Densities)¶ Let $\lambda\in\mathcal{P}(\mathbb{R}^{n})$ have a density $f$ with respect to $\tilde{\gamma}_{n}$. Then $f\circ r_{n}$ is a density of $E_{n}(\lambda)$ with respect to $\gamma_{c}$.

7. (Relative entropy)¶ A measure $\lambda\in\mathcal{P}(\mathbb{R}^{n})$ has finite relative entropy with respect to $\tilde{\gamma}_{n}$ if and only if $E_{n}(\lambda)$ has finite relative entropy with respect to $\gamma_{c}$, and then $H(E_{n}(\lambda)\,|\,\gamma_{c})=H(\lambda\,|\,\tilde{\gamma}_{n})$.

8. (A Gaussian head with an arbitrary tail)¶ Let $\sigma\in\mathcal{P}(X)$ have finite relative entropy with respect to $\tau_{n}$. The product measure $\tilde{\gamma}_{n}\otimes\sigma$ on $\mathcal{B}(\mathbb{R}^{n})\otimes\mathcal{B}(X)$ exists, both factors being finite, and is a probability measure, its value on $\mathbb{R}^{n}\times X$ being $\tilde{\gamma}_{n}(\mathbb{R}^{n})\,\sigma(X)=1$; and $\Psi_{n}$ is Borel by The Gaussian-Tail Extension of a Probability Measure on the Rescaled Head §borel. Then $(\Psi_{n})_{\#}(\tilde{\gamma}_{n}\otimes\sigma)$ has finite relative entropy with respect to $\gamma_{c}$, and

$$H\bigl((\Psi_{n})_{\#}(\tilde{\gamma}_{n}\otimes\sigma)\,\big|\,\gamma_{c}\bigr)\le H(\sigma\,|\,\tau_{n}).$$

9. (Relative entropy of the rescaled heads)¶ Let $\mu\in\mathcal{P}(X)$ have finite relative entropy with respect to $\gamma_{c}$. Then for every $m\in\mathbb{N}$ the measure $\tilde{\mu}_{m}$ has finite relative entropy with respect to $\tilde{\gamma}_{m}$ and $H(\tilde{\mu}_{m}\,|\,\tilde{\gamma}_{m})=H((p_{m})_{\#}\mu\,|\,\gamma_{c^{(m)}})$; the sequence $\bigl(H(\tilde{\mu}_{m}\,|\,\tilde{\gamma}_{m})\bigr)_{m\in\mathbb{N}}$ is nondecreasing and converges to $H(\mu\,|\,\gamma_{c})$.

10. (Relative score)¶ Let $\lambda\in\mathcal{P}_{2}(\mathbb{R}^{n})$ have finite Fisher information relative to $\tilde{\gamma}_{n}$, and for $k\in[n]$ let $g_{k}:\mathbb{R}^{n}\to\mathbb{R}$ be a Borel representative of the component $(\zeta^{\tilde{c}^{(n)}}_{\lambda})_{k}\in L^{2}(\lambda)$. Then $E_{n}(\lambda)$, which lies in $\mathcal{P}_{2}(X)$ by claim 5, has a relative score $(\zeta_{k})_{k\in\mathbb{N}}$ with respect to $\gamma_{c}$, where for $k\in[n]$ the component $\zeta_{k}$ is the class of $a_{k}^{-1/2}(g_{k}\circ r_{n})$, which is square-integrable with respect to $E_{n}(\lambda)$ with a class independent of the choice of $g_{k}$ by claim 3 and the change-of-variables formula of Borel Probability Measures on a Real Hilbert Space with an Orthonormal Basis: Standing Notation §pushforward, and where $\zeta_{k}=0$ for $k>n$.

11. (Fisher information)¶ In the situation of claim 10, $E_{n}(\lambda)$ has finite Fisher information relative to $\gamma_{c}$ with weights $a$, and $\mathcal{I}_{a}(E_{n}(\lambda)\,|\,\gamma_{c})=\mathcal{I}(\lambda\,|\,\tilde{\gamma}_{n})$.

12. (Noise Wasserstein distance)¶ For $\lambda,\lambda'\in\mathcal{P}_{2}(\mathbb{R}^{n})$, the measures $E_{n}(\lambda)$ and $E_{n}(\lambda')$ belong to $\mathcal{P}^{a}_{\rho}$, and

$$W_{a}\bigl(E_{n}(\lambda),E_{n}(\lambda')\bigr)\le W_{2}(\lambda,\lambda').$$