Proof of Talagrand's Inequality in the Noise Norm for a Diagonal Gaussian Reference Measure on a Hilbert Space

Each result cited is universally quantified over the data in its own statement.

Throughout, $c$, $\rho=\gamma_{c}$, $\kappa$, $a$ and $\mu$ are as in the statement, and $X$, its inner product $\langle\cdot,\cdot\rangle$, the orthonormal basis $(e_{k})_{k\in\mathbb{N}}$, the coordinates $x_{k}=\langle x,e_{k}\rangle$, and, for $n\in\mathbb{N}$, the maps $p_{n}$, $p_{n}^{*}$, $P_{n}=p_{n}^{*}\circ p_{n}$ and $Q_{n}=\mathrm{id}_{X}-P_{n}$ are those of Borel Probability Measures on a Real Hilbert Space with an Orthonormal Basis: Standing Notation §coordinates, in force through Probability Measures on a Hilbert Space Transported in the Noise Norm: Standing Notation §background. The noise space $X^{a}$, its norm $|\cdot|_{a}$, the Borel function $n_{a}$ of The Noise Space is a Real Hilbert Space: Orthonormal Basis, Continuous Embedding, Partial Sums, Closed Balls and Borel Measurability §borel, the sets $\Pi^{a}(\cdot,\cdot)$ and the noise cost $I^{a}$ are those of Probability Measures on a Hilbert Space Transported in the Noise Norm: Standing Notation §noise-space and Couplings of Finite Noise Cost and Their Noise Cost. By Relative Entropy on a Measurable Space: the Gibbs Inequality, the Variational Criterion and Formula, the Entropy Inequality, Small Sets and Data Processing §gibbs, $H(\mu\,|\,\gamma_{c})$ is a nonnegative real number.

Step 1 (The approximating measures). By Relative Entropy with Respect to a Diagonal Gaussian Measure on a Hilbert Space: the Moment Bound, the Cutoff Projections, and Bounded, Tight, Weakly Closed, Wasserstein-Closed and Weakly Sequentially Compact Sublevel Sets §moment, $\mu\in\mathcal{P}_{2}(X)$. So for every $n\in\mathbb{N}$ the measure

$$\mu^{(n)}=(\Phi_{n})_{\#}\bigl((p_{n})_{\#}\mu\otimes(Q_{n})_{\#}\gamma_{c}\bigr)\in\mathcal{P}(X)$$

of Replacing the Tail of a Probability Measure on a Hilbert Space by an Independent Diagonal Gaussian Tail: Same Head and Convergence in the Quadratic Wasserstein Distance is defined, with $\Phi_{n}:\mathbb{R}^{n}\times X\to X$, $\Phi_{n}(u,w)=p_{n}^{*}(u)+w$, the Borel map of Head and Tail of a Diagonal Gaussian Measure on a Hilbert Space are Independent §synthesis; here $\mathbb{R}^{n}\times X$ carries the product metric, whose Borel $\sigma$-algebra is $\mathcal{B}(\mathbb{R}^{n})\otimes\mathcal{B}(X)$ by The Borel Sigma-Algebra of a Product of Two Separable Metric Spaces is the Product Sigma-Algebra §product, both factors being separable (Euclidean Space is a Separable Metric Space §separable and Borel Probability Measures on a Real Hilbert Space with an Orthonormal Basis: Standing Notation §space). Fix $n\in\mathbb{N}$ until the end of Step 6, and write $\mu_{n}=(p_{n})_{\#}\mu$ and $\nu^{0}=(Q_{n})_{\#}\gamma_{c}$, Borel probability measures on $\mathbb{R}^{n}$ and on $X$.

Step 2 (A Euclidean transport map for the head). By Diagonal Gaussian Measures on a Hilbert Space §measure, $(p_{n})_{\#}\gamma_{c}=\gamma_{c^{(n)}}$, where $c^{(n)}$ is the truncation of $c$, a variance vector in $\mathbb{R}^{n}$, and $\gamma_{c^{(n)}}$ is the diagonal Gaussian measure on $\mathbb{R}^{n}$ with variances $c^{(n)}$. By Relative Entropy of the Finite-Dimensional Projections of Borel Probability Measures on a Hilbert Space: Monotonicity and Approximation §projections, applied with $\gamma_{c}$ in place of $\gamma$, the probability measure $\mu_{n}$ on $(\mathbb{R}^{n},\mathcal{B}(\mathbb{R}^{n}))$ has finite relative entropy with respect to $\gamma_{c^{(n)}}$ and $H(\mu_{n}\,|\,\gamma_{c^{(n)}})\le H(\mu\,|\,\gamma_{c})$. Now Talagrand's Inequality for a Diagonal Gaussian Measure on Euclidean Space in the Cameron-Martin Cost §talagrand, read with $n$ in place of $d$, $c^{(n)}$ in place of $c$ and $\mu_{n}$ in place of $\mu$, gives a Borel map $T^{(n)}:\mathbb{R}^{n}\to\mathbb{R}^{n}$, with coordinate functions $T^{(n)}_{k}$, such that $(T^{(n)})_{\#}\gamma_{c^{(n)}}=\mu_{n}$ and the nonnegative Borel function

$$g_{n}:\mathbb{R}^{n}\to\mathbb{R},\qquad g_{n}(u)=\sum_{k=1}^{n}\frac{\bigl(T^{(n)}_{k}(u)-u_{k}\bigr)^{2}}{c_{k}},$$

is integrable with respect to $\gamma_{c^{(n)}}$ with

$$\int_{\mathbb{R}^{n}}g_{n}\,d\gamma_{c^{(n)}}\le2\,H(\mu_{n}\,|\,\gamma_{c^{(n)}})\le2\,H(\mu\,|\,\gamma_{c}).\tag{1}$$

Step 3 (The transport map on $X$). Let $F_{n}:\mathbb{R}^{n}\times X\to\mathbb{R}^{n}\times X$ be the map $F_{n}(u,w)=(T^{(n)}(u),w)$. For $A\in\mathcal{B}(\mathbb{R}^{n})$ and $B\in\mathcal{B}(X)$ one has $F_{n}^{-1}(A\times B)=(T^{(n)})^{-1}(A)\times B$, a measurable rectangle, $T^{(n)}$ being Borel. Since the measurable rectangles generate $\mathcal{B}(\mathbb{R}^{n})\otimes\mathcal{B}(X)$ (Product Sigma-Algebra), $F_{n}$ is measurable by claim 2 of Generator Criterion for Measurability. Let $h_{n}:X\to\mathbb{R}^{n}\times X$, $h_{n}(x)=(p_{n}(x),Q_{n}x)$, be the Borel map of Head and Tail of a Diagonal Gaussian Measure on a Hilbert Space are Independent §head-tail, and define

$$U_{n}=\Phi_{n}\circ F_{n}\circ h_{n}:X\to X,\qquad U_{n}(x)=p_{n}^{*}\bigl(T^{(n)}(p_{n}(x))\bigr)+Q_{n}x .$$

It is Borel by claim 4 of Borel Measurability and Bounded Integration on a Metric Space, applied twice (recall from Step 1 that $\mathcal{B}(\mathbb{R}^{n}\times X)=\mathcal{B}(\mathbb{R}^{n})\otimes\mathcal{B}(X)$).

Step 4 (The image of $\gamma_{c}$). For measurable maps $G$ and $F$ whose composition $F\circ G$ is defined and a measure $\lambda$ on the domain of $G$, the definition of image measures in claim 1 of Image Measures, Measures with Densities, and Change of Variables gives $(F\circ G)_{\#}\lambda(B)=\lambda(G^{-1}(F^{-1}(B)))=F_{\#}(G_{\#}\lambda)(B)$ for every measurable $B$, that is, $(F\circ G)_{\#}\lambda=F_{\#}(G_{\#}\lambda)$. By Head and Tail of a Diagonal Gaussian Measure on a Hilbert Space are Independent §head-tail, $(h_{n})_{\#}\gamma_{c}=\gamma_{c^{(n)}}\otimes\nu^{0}$. For $A\in\mathcal{B}(\mathbb{R}^{n})$ and $B\in\mathcal{B}(X)$, by Step 3, the defining property of the product measure (Existence and Uniqueness of the Product Measure) and Step 2,

$$(F_{n})_{\#}\bigl(\gamma_{c^{(n)}}\otimes\nu^{0}\bigr)(A\times B)=\gamma_{c^{(n)}}\bigl((T^{(n)})^{-1}(A)\bigr)\,\nu^{0}(B)=\mu_{n}(A)\,\nu^{0}(B).$$

The image measure $(F_{n})_{\#}(\gamma_{c^{(n)}}\otimes\nu^{0})$ is a measure on $\mathcal{B}(\mathbb{R}^{n})\otimes\mathcal{B}(X)$ by claim 1 of Image Measures, Measures with Densities, and Change of Variables, and $\mu_{n}$ and $\nu^{0}$ are finite, hence $\sigma$-finite; so the uniqueness in Existence and Uniqueness of the Product Measure gives $(F_{n})_{\#}(\gamma_{c^{(n)}}\otimes\nu^{0})=\mu_{n}\otimes\nu^{0}$. Consequently

$$(U_{n})_{\#}\gamma_{c}=(\Phi_{n})_{\#}\Bigl((F_{n})_{\#}\bigl((h_{n})_{\#}\gamma_{c}\bigr)\Bigr)=(\Phi_{n})_{\#}\bigl(\mu_{n}\otimes\nu^{0}\bigr)=\mu^{(n)} .$$

Step 5 (The displacement lies in the noise space). Let $x\in X$ and put $v=T^{(n)}(p_{n}(x))-p_{n}(x)\in\mathbb{R}^{n}$. Since $Q_{n}x=x-p_{n}^{*}(p_{n}(x))$ and $p_{n}^{*}$, being given by the finite sum $p_{n}^{*}(y)=\sum_{k=1}^{n}y_{k}e_{k}$, is linear,

$$U_{n}(x)-x=p_{n}^{*}\bigl(T^{(n)}(p_{n}(x))\bigr)-p_{n}^{*}\bigl(p_{n}(x)\bigr)=p_{n}^{*}(v).$$

The coordinates of $y=p_{n}^{*}(v)$ are $y_{k}=v_{k}$ for $k\le n$, because $p_{n}(p_{n}^{*}(v))=v$ by Borel Sets of a Hilbert Space with an Orthonormal Basis: Coordinates, Determination by Finite-Dimensional Projections, and Pairs §continuity, and $y_{k}=\sum_{j=1}^{n}v_{j}\langle e_{j},e_{k}\rangle=0$ for $k>n$, by the orthonormality of the basis (Orthonormal Basis of a Real Hilbert Space §basis). Hence, for $N\in\mathbb{N}$, the partial sum $\sum_{k=1}^{N}a_{k}^{-1}y_{k}^{2}$ of The Noise Space is a Real Hilbert Space: Orthonormal Basis, Continuous Embedding, Partial Sums, Closed Balls and Borel Measurability §partial-sums equals $\sum_{k=1}^{\min(N,n)}a_{k}^{-1}v_{k}^{2}$, which is at most $\sum_{k=1}^{n}a_{k}^{-1}v_{k}^{2}$ and equals it for $N=n$. By The Noise Space is a Real Hilbert Space: Orthonormal Basis, Continuous Embedding, Partial Sums, Closed Balls and Borel Measurability §partial-sums, $y\in X^{a}$ and $|y|_{a}^{2}$, the supremum of these partial sums, is $\sum_{k=1}^{n}a_{k}^{-1}v_{k}^{2}$. Since $c_{k}\le\kappa\,a_{k}$ and $a_{k},c_{k}>0$, one has $a_{k}^{-1}\le\kappa\,c_{k}^{-1}$ for every $k$; and $v_{k}=T^{(n)}_{k}(p_{n}(x))-(p_{n}(x))_{k}$. So, by the definition of $n_{a}$ in The Noise Space is a Real Hilbert Space: Orthonormal Basis, Continuous Embedding, Partial Sums, Closed Balls and Borel Measurability §borel,

$$U_{n}(x)-x\in X^{a}\qquad\text{and}\qquad n_{a}\bigl(U_{n}(x)-x\bigr)=\sum_{k=1}^{n}\frac{v_{k}^{2}}{a_{k}}\le\kappa\,g_{n}\bigl(p_{n}(x)\bigr)\qquad\text{for every }x\in X.\tag{2}$$

Step 6 (A coupling of finite noise cost). The function $g_{n}\circ p_{n}$ is Borel and nonnegative, $p_{n}$ being Borel by Borel Sets of a Hilbert Space with an Orthonormal Basis: Coordinates, Determination by Finite-Dimensional Projections, and Pairs §continuity. By the change-of-variables formula of Borel Probability Measures on a Real Hilbert Space with an Orthonormal Basis: Standing Notation §pushforward and $(p_{n})_{\#}\gamma_{c}=\gamma_{c^{(n)}}$, one has $\int_{X}g_{n}(p_{n}(x))\,\gamma_{c}(dx)=\int_{\mathbb{R}^{n}}g_{n}\,d\gamma_{c^{(n)}}$. By (2), monotonicity and homogeneity of the integral of nonnegative functions (Linearity and Monotonicity of the Lebesgue Integral §nonnegative) and (1),

$$\int_{X}n_{a}\bigl(U_{n}(x)-x\bigr)\,\gamma_{c}(dx)\le\kappa\int_{\mathbb{R}^{n}}g_{n}\,d\gamma_{c^{(n)}}\le2\kappa\,H(\mu\,|\,\gamma_{c})<\infty .$$

Since $U_{n}$ is Borel with $U_{n}(x)-x\in X^{a}$ for every $x$ (Steps 3 and 5), Couplings of Finite Noise Cost: the Support Bound, Swap, Displacement Couplings and Lower Semicontinuity of the Noise Cost §displacement, applied with $\gamma_{c}$ in place of $\mu$ and $U_{n}$ in place of $S$, together with Step 4, gives $\Gamma_{n}=(\mathrm{id}_{X},U_{n})_{\#}\gamma_{c}\in\Pi^{a}(\gamma_{c},\mu^{(n)})$ with $I^{a}(\Gamma_{n})\le2\kappa\,H(\mu\,|\,\gamma_{c})$. By Couplings of Finite Noise Cost: the Support Bound, Swap, Displacement Couplings and Lower Semicontinuity of the Noise Cost §swap, with $\sigma$ the swap map there, $\sigma_{\#}\Gamma_{n}\in\Pi^{a}(\mu^{(n)},\rho)$ and $I^{a}(\sigma_{\#}\Gamma_{n})=I^{a}(\Gamma_{n})$. Hence the ordered pair $(\mu^{(n)},\rho)$ is noise-connected (Couplings of Finite Noise Cost and Their Noise Cost §connected), that is, $\mu^{(n)}\in\mathcal{P}^{a}_{\rho}$ by The Measures Noise-Connected to the Reference Measure §space, and by The Noise Wasserstein Distance §distance

$$W_{a}(\mu^{(n)},\rho)^{2}\le I^{a}(\sigma_{\#}\Gamma_{n})\le2\kappa\,H(\mu\,|\,\gamma_{c}),\qquad\text{so}\qquad0\le W_{a}(\mu^{(n)},\rho)\le\sqrt{2\kappa\,H(\mu\,|\,\gamma_{c})}.\tag{3}$$

Step 7 (Passage to the limit). Now let $n$ vary. By Replacing the Tail of a Probability Measure on a Hilbert Space by an Independent Diagonal Gaussian Tail: Same Head and Convergence in the Quadratic Wasserstein Distance §convergence, which applies since $\mu\in\mathcal{P}_{2}(X)$ (Step 1), $\mu^{(n)}\Rightarrow\mu$. The constant sequence with every term $\rho$ converges weakly to $\rho$, its integrals against each bounded continuous function being constant (Weak Convergence of Finite Borel Measures on a Metric Space), and $\rho\in\mathcal{P}^{a}_{\rho}$ by The Noise Wasserstein Distance is a Metric on the Measures Noise-Connected to the Reference Measure: Existence of Noise-Optimal Couplings, Comparison with the Quadratic Wasserstein Distance and Lower Semicontinuity §reference. By Step 6 every $\mu^{(n)}$ belongs to $\mathcal{P}^{a}_{\rho}$, and by (3) the sequence $(W_{a}(\mu^{(n)},\rho))_{n\in\mathbb{N}}$ is bounded. So The Noise Wasserstein Distance is a Metric on the Measures Noise-Connected to the Reference Measure: Existence of Noise-Optimal Couplings, Comparison with the Quadratic Wasserstein Distance and Lower Semicontinuity §lsc, applied with $\mu_{j}=\mu^{(j)}$, $\nu_{j}=\rho$ and $\nu=\rho$, shows that $(\mu,\rho)$ is noise-connected and that

$$W_{a}(\mu,\rho)\le\liminf_{n}W_{a}(\mu^{(n)},\rho)\le\sqrt{2\kappa\,H(\mu\,|\,\gamma_{c})};$$

the last inequality holds because, in Limit Inferior of a Bounded Sequence of Real Numbers, each infimum of a tail of the sequence is at most the first term of that tail, hence at most $\sqrt{2\kappa\,H(\mu\,|\,\gamma_{c})}$ by (3), and so is their supremum. Since $(\mu,\rho)$ is noise-connected, $\mu\in\mathcal{P}^{a}_{\rho}$ by The Measures Noise-Connected to the Reference Measure §space, which is claim 1. Squaring the last display, both sides being nonnegative, and recalling $\rho=\gamma_{c}$ gives $W_{a}(\mu,\gamma_{c})^{2}\le2\kappa\,H(\mu\,|\,\gamma_{c})$, which is claim 2.