Proof of Tail Replacement in the Noise Wasserstein Distance: a Measure of Finite Relative Entropy is Approximated by the Gaussian-Tail Extensions of Its Rescaled Heads

Each result cited is universally quantified over the data in its own statement. The setting A Diagonal Gaussian Reference Measure on the Noise Wasserstein Space, Rescaled Heads and Gaussian Tails: Standing Notation is in force, so the reference measure is $\rho=\gamma_{c}$ and the setting Probability Measures on a Hilbert Space Transported in the Noise Norm: Standing Notation is in force through A Diagonal Gaussian Reference Measure on the Noise Wasserstein Space, Rescaled Heads and Gaussian Tails: Standing Notation §background; with the given $\kappa$, the hypotheses of Talagrand's Inequality in the Noise Norm for a Diagonal Gaussian Reference Measure on a Hilbert Space on $c$, $\kappa$ and $a$ hold, so that theorem applies to every member of $\mathcal{P}(X)$ of finite relative entropy with respect to $\gamma_{c}$. Integrals of nonnegative measurable functions are added, scaled and compared by claim 1 of Linearity and Monotonicity of the Lebesgue Integral, integrals against image measures are computed by the change of variables formula of claim 2 of Image Measures, Measures with Densities, and Change of Variables, and image measures are those of claim 1 of that lemma; these tools are used without further mention. Throughout, $n\in\mathbb{N}$ is fixed, $[n]=\{1,\dots,n\}$, and for $x\in X$ we write $x_{k}=\langle x,e_{k}\rangle$; $S_{N}(x)=\sum_{k=1}^{N}a_{k}^{-1}x_{k}^{2}$ is the partial sum of The Noise Space is a Real Hilbert Space: Orthonormal Basis, Continuous Embedding, Partial Sums, Closed Balls and Borel Measurability §partial-sums, and $n_{a}$, $D_{a}$ and $c_{a}$ are those of The Noise Space is a Real Hilbert Space: Orthonormal Basis, Continuous Embedding, Partial Sums, Closed Balls and Borel Measurability §borel and The Noise Space is a Real Hilbert Space: Orthonormal Basis, Continuous Embedding, Partial Sums, Closed Balls and Borel Measurability §pairs, so that $c_{a}(x,y)=n_{a}(y-x)$.

Step 1 (Membership; claim 1). By Talagrand's Inequality in the Noise Norm for a Diagonal Gaussian Reference Measure on a Hilbert Space §connected, $\mu\in\mathcal{P}^{a}_{\rho}$. Hence $\mu\in\mathcal{P}_{2}(X)$ by The Noise Wasserstein Distance is a Metric on the Measures Noise-Connected to the Reference Measure: Existence of Noise-Optimal Couplings, Comparison with the Quadratic Wasserstein Distance and Lower Semicontinuity §moments, and then $\tilde{\mu}_{n}=(r_{n})_{\#}\mu\in\mathcal{P}_{2}(\mathbb{R}^{n})$ by Lifting Euclidean Tangent Fields of the Rescaled Head to Noise Tangent Fields on a Hilbert Space §head, which also shows that $r_{n}$ is continuous, hence Borel. Moreover $\tilde{\gamma}_{n}\in\mathcal{P}_{2}(\mathbb{R}^{n})$ by Gaussian-Tail Extensions: the Gaussian Reference Measure, Marginals, Densities, Relative Entropy, Relative Score and the Noise Wasserstein Distance §head-law. Applying Gaussian-Tail Extensions: the Gaussian Reference Measure, Marginals, Densities, Relative Entropy, Relative Score and the Noise Wasserstein Distance §distance with $\lambda=\tilde{\mu}_{n}$ and $\lambda'=\tilde{\gamma}_{n}$ gives $E_{n}(\tilde{\mu}_{n})\in\mathcal{P}^{a}_{\rho}$. Since $\mu$ and $E_{n}(\tilde{\mu}_{n})$ both belong to $\mathcal{P}^{a}_{\rho}$, the pair $(\mu,E_{n}(\tilde{\mu}_{n}))$ is noise-connected by The Noise Wasserstein Distance is a Metric on the Measures Noise-Connected to the Reference Measure: Existence of Noise-Optimal Couplings, Comparison with the Quadratic Wasserstein Distance and Lower Semicontinuity §connected, so $W_{a}(\mu,E_{n}(\tilde{\mu}_{n}))$ is defined by The Noise Wasserstein Distance §distance, and $W_{a}(\mu,E_{n}(\tilde{\mu}_{n}))^{2}\le I^{a}(\pi)$ for every $\pi\in\Pi^{a}(\mu,E_{n}(\tilde{\mu}_{n}))$. This proves claim 1.

Step 2 (Coordinate identities). Let $Z_{n}=\{w\in X:w_{k}=0\text{ for every }k\in[n]\}$. It is a linear subspace of $X$, and it is Borel, being the intersection of the preimages of the closed set $\{0\}$ under the finitely many coordinate functions $x\mapsto x_{k}$, $k\in[n]$, which are continuous by Borel Sets of a Hilbert Space with an Orthonormal Basis: Coordinates, Determination by Finite-Dimensional Projections, and Pairs §continuity. We record six facts.

(a) For $w\in X$, $P_{n}w=p_{n}^{*}(p_{n}(w))=\sum_{j=1}^{n}w_{j}e_{j}$ by Borel Probability Measures on a Real Hilbert Space with an Orthonormal Basis: Standing Notation §coordinates, so by orthonormality $(P_{n}w)_{k}=w_{k}$ for $k\in[n]$ and $(P_{n}w)_{k}=0$ for $k>n$; hence $(Q_{n}w)_{k}=0$ for $k\in[n]$ and $(Q_{n}w)_{k}=w_{k}$ for $k>n$. Consequently $Q_{n}w\in Z_{n}$ for every $w\in X$; if $w\in Z_{n}$ then $p_{n}(w)=0$, so $P_{n}w=0$ and $Q_{n}w=w$; and $Q_{n}\circ Q_{n}=Q_{n}$. Likewise, for $\zeta\in\mathbb{R}^{n}$ the vector $p_{n}^{*}(\zeta)=\sum_{j=1}^{n}\zeta_{j}e_{j}$ has $k$-th coordinate $\zeta_{k}$ for $k\in[n]$ and $0$ for $k>n$.

(b) For $u\in\mathbb{R}^{n}$ and $w\in X$, $\Psi_{n}(u,w)=p_{n}^{*}(\eta)+w$ with $\eta=(a_{1}^{1/2}u_{1},\dots,a_{n}^{1/2}u_{n})$. The maps $p_{n}$, $p_{n}^{*}$, $P_{n}$ and $Q_{n}$ are linear, and $p_{n}(p_{n}^{*}(\eta))=\eta$ by Borel Sets of a Hilbert Space with an Orthonormal Basis: Coordinates, Determination by Finite-Dimensional Projections, and Pairs §continuity, so $Q_{n}p_{n}^{*}(\eta)=p_{n}^{*}(\eta)-p_{n}^{*}(p_{n}(p_{n}^{*}(\eta)))=0$. Hence $Q_{n}\Psi_{n}(u,w)=Q_{n}w$. If moreover $w\in Z_{n}$, then $Q_{n}\Psi_{n}(u,w)=w$ by (a), and $p_{n}(\Psi_{n}(u,w))=\eta+p_{n}(w)=\eta$, so that $r_{n}(\Psi_{n}(u,w))=(a_{1}^{-1/2}\eta_{1},\dots,a_{n}^{-1/2}\eta_{n})=u$ by the formula for $r_{n}$ in Lifting Euclidean Tangent Fields of the Rescaled Head to Noise Tangent Fields on a Hilbert Space.

(c) For $x\in X$, $a_{k}^{1/2}(r_{n}(x))_{k}=a_{k}^{1/2}a_{k}^{-1/2}x_{k}=x_{k}$ for $k\in[n]$, so $\Psi_{n}(r_{n}(x),Q_{n}x)=p_{n}^{*}(p_{n}(x))+Q_{n}x=P_{n}x+Q_{n}x=x$.

(d) Let $z\in X^{a}$. By (a), $S_{N}(Q_{n}z)=\sum_{n<k\le N}a_{k}^{-1}z_{k}^{2}\le S_{N}(z)\le|z|_{a}^{2}$ for every $N\in\mathbb{N}$, the last inequality by The Noise Space is a Real Hilbert Space: Orthonormal Basis, Continuous Embedding, Partial Sums, Closed Balls and Borel Measurability §partial-sums. By the same claim, $Q_{n}z\in X^{a}$ and $|Q_{n}z|_{a}^{2}=\sup_{N}S_{N}(Q_{n}z)\le|z|_{a}^{2}$.

(e) Let $x\in X$, $v\in\mathbb{R}^{n}$ and $w'\in Z_{n}$ be such that $t=w'-Q_{n}x\in X^{a}$, and let $y=\Psi_{n}(v,w')$. Then $y-x\in X^{a}$ and $|y-x|_{a}^{2}=\lVert v-r_{n}(x)\rVert^{2}+|t|_{a}^{2}$; that is, $(x,y)\in D_{a}$ and $c_{a}(x,y)=\lVert v-r_{n}(x)\rVert^{2}+c_{a}(Q_{n}x,w')$. Indeed, by (b) and (c), $y-x=p_{n}^{*}(\eta-p_{n}(x))+t$ with $\eta_{k}=a_{k}^{1/2}v_{k}$, and $t\in Z_{n}$ because $Z_{n}$ is a linear subspace containing $w'$ and $Q_{n}x$. By (a) and (c), the $k$-th coordinate of $y-x$ is $a_{k}^{1/2}(v_{k}-(r_{n}(x))_{k})$ for $k\in[n]$ and $t_{k}$ for $k>n$. Hence for $N\ge n$, $S_{N}(y-x)=\lVert v-r_{n}(x)\rVert^{2}+S_{N}(t)$, since $S_{N}(t)$ has no contribution from $k\in[n]$. By The Noise Space is a Real Hilbert Space: Orthonormal Basis, Continuous Embedding, Partial Sums, Closed Balls and Borel Measurability §partial-sums, $S_{N}(t)\le|t|_{a}^{2}$ with $\sup_{N}S_{N}(t)=|t|_{a}^{2}$, and $(S_{N}(y-x))_{N}$ is nondecreasing, so it is bounded above, $y-x\in X^{a}$, and $|y-x|_{a}^{2}=\sup_{N\ge n}S_{N}(y-x)=\lVert v-r_{n}(x)\rVert^{2}+|t|_{a}^{2}$.

(f) $\tau_{n}(Z_{n})=\gamma_{c}(Q_{n}^{-1}(Z_{n}))=\gamma_{c}(X)=1$ by (a) and A Diagonal Gaussian Reference Measure on the Noise Wasserstein Space, Rescaled Heads and Gaussian Tails: Standing Notation §tails.

Step 3 (The Gaussian reference measure on head cells). We show that for all $C\in\mathcal{B}(\mathbb{R}^{n})$ and $B\in\mathcal{B}(X)$,

$$\gamma_{c}\bigl(r_{n}^{-1}(C)\cap Q_{n}^{-1}(B)\bigr)=\tilde{\gamma}_{n}(C)\,\tau_{n}(B).$$

By Gaussian-Tail Extensions: the Gaussian Reference Measure, Marginals, Densities, Relative Entropy, Relative Score and the Noise Wasserstein Distance §reconstruction and The Gaussian-Tail Extension of a Probability Measure on the Rescaled Head §extension, $\gamma_{c}=(\Psi_{n})_{\#}(\tilde{\gamma}_{n}\otimes\tau_{n})$, with $\Psi_{n}$ measurable for $\mathcal{B}(\mathbb{R}^{n})\otimes\mathcal{B}(X)$ by The Gaussian-Tail Extension of a Probability Measure on the Rescaled Head §borel. Let $E=r_{n}^{-1}(C)\cap Q_{n}^{-1}(B)$, a Borel set as $r_{n}$ and $Q_{n}$ are Borel. By (b) of Step 2, a point $(u,w)$ with $w\in Z_{n}$ lies in $\Psi_{n}^{-1}(E)$ if and only if $u\in C$ and $w\in B$; so $\Psi_{n}^{-1}(E)\cap(\mathbb{R}^{n}\times Z_{n})=C\times(B\cap Z_{n})$. The rectangle $\mathbb{R}^{n}\times(X\setminus Z_{n})$ has $(\tilde{\gamma}_{n}\otimes\tau_{n})$-measure $\tilde{\gamma}_{n}(\mathbb{R}^{n})\,\tau_{n}(X\setminus Z_{n})=0$ by Existence and Uniqueness of the Product Measure and (f) of Step 2. Hence, by additivity of the measure over $\Psi_{n}^{-1}(E)\cap(\mathbb{R}^{n}\times Z_{n})$ and $\Psi_{n}^{-1}(E)\setminus(\mathbb{R}^{n}\times Z_{n})$, the latter having measure $0$ by monotonicity,

$$\gamma_{c}(E)=(\tilde{\gamma}_{n}\otimes\tau_{n})\bigl(C\times(B\cap Z_{n})\bigr)=\tilde{\gamma}_{n}(C)\,\tau_{n}(B\cap Z_{n})=\tilde{\gamma}_{n}(C)\,\tau_{n}(B),$$

using Existence and Uniqueness of the Product Measure and $\tau_{n}(B\setminus Z_{n})=0$. With $B=X$ this gives $\gamma_{c}(r_{n}^{-1}(C))=\tilde{\gamma}_{n}(C)$.

Step 4 (The order of choices). Fix a positive real number $\varepsilon$. We make the following choices, in this order; all later constructions involve no further choice of numbers.

(i) A radius $R$. For $m\in\mathbb{N}$ let $f_{m}(u)=\lVert u\rVert^{2}\mathbf{1}_{\{\lVert u\rVert\le m\}}$ on $\mathbb{R}^{n}$; these are nonnegative Borel functions, nondecreasing in $m$, with pointwise supremum $\lVert u\rVert^{2}$. By Monotone Convergence Theorem, $\int f_{m}\,d\tilde{\mu}_{n}$ increases to the second moment $M_{2}(\tilde{\mu}_{n})=\int\lVert u\rVert^{2}\,\tilde{\mu}_{n}(du)$ of The Second Moment of a Probability Measure on Euclidean Space and the Probability Measures with Finite Second Moment §moment, which is finite by Step 1 and The Second Moment of a Probability Measure on Euclidean Space and the Probability Measures with Finite Second Moment §space. Choose $R=m\in\mathbb{N}$ with $\int f_{m}\,d\tilde{\mu}_{n}>M_{2}(\tilde{\mu}_{n})-\varepsilon$; since $\lVert u\rVert^{2}=f_{R}(u)+\lVert u\rVert^{2}\mathbf{1}_{U}(u)$ with $U=\{u\in\mathbb{R}^{n}:\lVert u\rVert>R\}$, we get

$$\int_{\mathbb{R}^{n}}\lVert u\rVert^{2}\mathbf{1}_{U}(u)\,\tilde{\mu}_{n}(du)<\varepsilon .$$

(ii) A partition $\mathcal{C}'$. By Gaussian-Tail Extensions: the Gaussian Reference Measure, Marginals, Densities, Relative Entropy, Relative Score and the Noise Wasserstein Distance §heads (with $m=n$), $\tilde{\mu}_{n}$ has finite relative entropy with respect to $\tilde{\gamma}_{n}$. By Relative Entropy over a Finite Partition: the Chain Rule, Images of the Pieces, Refinement and Approximation by Partition Entropies §approximation, applied on $(\mathbb{R}^{n},\mathcal{B}(\mathbb{R}^{n}))$ with $\nu=\tilde{\mu}_{n}$ and $\gamma=\tilde{\gamma}_{n}$, choose a finite measurable partition $\mathcal{C}'$ of $\mathbb{R}^{n}$ with $h_{\mathcal{C}'}(\tilde{\mu}_{n}\,|\,\tilde{\gamma}_{n})>H(\tilde{\mu}_{n}\,|\,\tilde{\gamma}_{n})-\varepsilon$, the partition entropy being defined by Relative Entropy over a Finite Partition: the Chain Rule, Images of the Pieces, Refinement and Approximation by Partition Entropies §chain-rule.

(iii) A partition $\mathcal{C}''$. Let $\delta=(\varepsilon/n)^{1/2}$ and choose $N\in\mathbb{N}$ with $R\le N\delta$. For $j\in\{-N,\dots,N\}^{n}$ let $K_{j}=\{u\in\mathbb{R}^{n}:j_{k}\delta\le u_{k}<(j_{k}+1)\delta\text{ for }k\in[n]\}\cap\{u:\lVert u\rVert\le R\}$. Each $K_{j}$ is Borel, being a finite intersection of closed and open sets (the coordinate functions and $\lVert\cdot\rVert$ are continuous). The sets $K_{j}$ are pairwise disjoint, because $j_{k}=\lfloor u_{k}/\delta\rfloor$ for $u\in K_{j}$, and they cover $\{\lVert u\rVert\le R\}$: if $\lVert u\rVert\le R$ then $|u_{k}|\le\lVert u\rVert\le N\delta$ by Elementary Properties of the Euclidean Norm on $\mathbb{R}^n$ §coordinate, so $j_{k}=\lfloor u_{k}/\delta\rfloor\in\{-N,\dots,N\}$. For $u,u'\in K_{j}$, $|u_{k}-u'_{k}|<\delta$ for each $k$, so $\lVert u-u'\rVert^{2}<n\delta^{2}=\varepsilon$. Thus $\mathcal{C}''$, consisting of the sets $K_{j}$ and the open set $U$, is a finite measurable partition of $\mathbb{R}^{n}$ each of whose cells either is contained in $U$ or has $\lVert u-u'\rVert^{2}<\varepsilon$ for all of its points $u,u'$.

(iv) The partition $\mathcal{C}$. Let $\mathcal{C}=(C_{1},\dots,C_{m})$ be an enumeration of the sets $C'\cap C''$ with $C'$ a cell of $\mathcal{C}'$ and $C''$ a cell of $\mathcal{C}''$. These are Borel, pairwise disjoint, and cover $\mathbb{R}^{n}$, so $\mathcal{C}$ is a finite measurable partition of $\mathbb{R}^{n}$ which refines both $\mathcal{C}'$ and $\mathcal{C}''$. By Relative Entropy over a Finite Partition: the Chain Rule, Images of the Pieces, Refinement and Approximation by Partition Entropies §refinement and (ii),

$$h_{\mathcal{C}}(\tilde{\mu}_{n}\,|\,\tilde{\gamma}_{n})\ge h_{\mathcal{C}'}(\tilde{\mu}_{n}\,|\,\tilde{\gamma}_{n})>H(\tilde{\mu}_{n}\,|\,\tilde{\gamma}_{n})-\varepsilon .$$

A nonempty cell $C_{i}$ lies in exactly one cell of $\mathcal{C}''$; we call $i$ small if that cell is some $K_{j}$, so that $\lVert u-u'\rVert^{2}<\varepsilon$ for $u,u'\in C_{i}$, and large if it is $U$, so that $C_{i}\subseteq U$.

Step 5 (Pulling the partition back to $X$). Let $A_{i}=r_{n}^{-1}(C_{i})$ for $i\in[m]$. Since $r_{n}$ is Borel and preimages preserve disjointness and unions, $\mathcal{A}=(A_{1},\dots,A_{m})$ is a finite measurable partition of $X$. Let $p_{i}=\mu(A_{i})=\tilde{\mu}_{n}(C_{i})$ and $q_{i}=\gamma_{c}(A_{i})=\tilde{\gamma}_{n}(C_{i})$, the latter by Step 3, and let $J=\{i\in[m]:0<p_{i}\}$; for $i\in J$ the cell $C_{i}$ is nonempty and $0<q_{i}$ by Relative Entropy over a Finite Partition: the Chain Rule, Images of the Pieces, Refinement and Approximation by Partition Entropies §chain-rule. The numbers $p_{i}$, $q_{i}$ being the same for $(\mu,\gamma_{c},\mathcal{A})$ and for $(\tilde{\mu}_{n},\tilde{\gamma}_{n},\mathcal{C})$, The Partition Entropy of a Probability Measure Relative to Another over a Finite Measurable Partition §partition-entropy gives $h_{\mathcal{A}}(\mu\,|\,\gamma_{c})=h_{\mathcal{C}}(\tilde{\mu}_{n}\,|\,\tilde{\gamma}_{n})$.

For $i\in J$ let $\mu_{i}=\mu(\cdot\,|\,A_{i})$ and $\gamma_{i}=\gamma_{c}(\cdot\,|\,A_{i})$ be the conditioned probability measures, which are the pieces of $\mu$ and of $\gamma_{c}$ on $A_{i}$ in Relative Entropy over a Finite Partition: the Chain Rule, Images of the Pieces, Refinement and Approximation by Partition Entropies. For $B\in\mathcal{B}(X)$, Step 3 gives $\gamma_{i}(Q_{n}^{-1}(B))=\gamma_{c}(A_{i}\cap Q_{n}^{-1}(B))/q_{i}=\tilde{\gamma}_{n}(C_{i})\tau_{n}(B)/\tilde{\gamma}_{n}(C_{i})=\tau_{n}(B)$; that is, $(Q_{n})_{\#}\gamma_{i}=\tau_{n}$. Let $\sigma_{i}=(Q_{n})_{\#}\mu_{i}\in\mathcal{P}(X)$. Applying Relative Entropy over a Finite Partition: the Chain Rule, Images of the Pieces, Refinement and Approximation by Partition Entropies §images on $(X,\mathcal{B}(X))$ with $\nu=\mu$, $\gamma=\gamma_{c}$, the partition $\mathcal{A}$ and the Borel map $R=Q_{n}$ (Borel by Borel Sets of a Hilbert Space with an Orthonormal Basis: Coordinates, Determination by Finite-Dimensional Projections, and Pairs §continuity), each $\sigma_{i}$, $i\in J$, has finite relative entropy with respect to $\tau_{n}$ and

$$\sum_{i\in J}p_{i}\,H(\sigma_{i}\,|\,\tau_{n})\le H(\mu\,|\,\gamma_{c})-h_{\mathcal{A}}(\mu\,|\,\gamma_{c})=H(\mu\,|\,\gamma_{c})-h_{\mathcal{C}}(\tilde{\mu}_{n}\,|\,\tilde{\gamma}_{n})<H(\mu\,|\,\gamma_{c})-H(\tilde{\mu}_{n}\,|\,\tilde{\gamma}_{n})+\varepsilon,\qquad(*)$$

the last inequality by Step 4 (iv).

For $i\in J$ let $\tilde{\mu}_{i}=(r_{n})_{\#}\mu_{i}\in\mathcal{P}(\mathbb{R}^{n})$. For $F\in\mathcal{B}(\mathbb{R}^{n})$, $p_{i}\tilde{\mu}_{i}(F)=\mu(r_{n}^{-1}(F)\cap A_{i})=\tilde{\mu}_{n}(F\cap C_{i})$, so $\tilde{\mu}_{i}$ is the conditioned probability measure $\tilde{\mu}_{n}(\cdot\,|\,C_{i})$, defined as $\tilde{\mu}_{n}(C_{i})=p_{i}>0$; in particular $\tilde{\mu}_{i}(C_{i})=1$, and $\tilde{\mu}_{i}$ is the measure with density $p_{i}^{-1}\mathbf{1}_{C_{i}}$ with respect to $\tilde{\mu}_{n}$ of claim 3 of Image Measures, Measures with Densities, and Change of Variables. Summing over $i$ and using that a cell with $p_{i}=0$ contributes $\tilde{\mu}_{n}(F\cap C_{i})\le p_{i}=0$, we get $\sum_{i\in J}p_{i}\tilde{\mu}_{i}(F)=\tilde{\mu}_{n}(F)$; likewise $\sum_{i\in J}p_{i}\mu_{i}(B)=\sum_{i\in J}\mu(B\cap A_{i})=\mu(B)$ for $B\in\mathcal{B}(X)$.

Finally, $\mu_{i}$, $\sigma_{i}$ and $\tau_{n}$ belong to $\mathcal{P}_{2}(X)$ for $i\in J$. Indeed, $\mu_{i}$ has density $p_{i}^{-1}\mathbf{1}_{A_{i}}$ with respect to $\mu$ by The Conditioned Probability Measure Given a Set of Positive Measure §conditioned, so by claim 3 of Image Measures, Measures with Densities, and Change of Variables, $M_{2}(\mu_{i})=p_{i}^{-1}\int\mathbf{1}_{A_{i}}|x|^{2}\,\mu(dx)\le p_{i}^{-1}M_{2}(\mu)<\infty$, as $\mu\in\mathcal{P}_{2}(X)$ by Step 1; and since $|Q_{n}x|\le|x|$ by Borel Sets of a Hilbert Space with an Orthonormal Basis: Coordinates, Determination by Finite-Dimensional Projections, and Pairs §continuity, $M_{2}(\sigma_{i})=\int|Q_{n}x|^{2}\,\mu_{i}(dx)\le M_{2}(\mu_{i})$ and $M_{2}(\tau_{n})=\int|Q_{n}x|^{2}\,\gamma_{c}(dx)\le M_{2}(\gamma_{c})<\infty$, with $\gamma_{c}\in\mathcal{P}_{2}(X)$ by A Diagonal Gaussian Reference Measure on the Noise Wasserstein Space, Rescaled Heads and Gaussian Tails: Standing Notation §gaussian; second moments are those of The Second Moment of a Borel Probability Measure on a Hilbert Space and the Probability Measures with Finite Second Moment §moment.

Step 6 (A tail coupling on each cell). Fix $i\in J$. Let $\alpha_{i}=(\Psi_{n})_{\#}(\tilde{\gamma}_{n}\otimes\sigma_{i})$. Since $\sigma_{i}$ has finite relative entropy with respect to $\tau_{n}$ (Step 5), Gaussian-Tail Extensions: the Gaussian Reference Measure, Marginals, Densities, Relative Entropy, Relative Score and the Noise Wasserstein Distance §tail-entropy shows that $\alpha_{i}$ has finite relative entropy with respect to $\gamma_{c}$ and $H(\alpha_{i}\,|\,\gamma_{c})\le H(\sigma_{i}\,|\,\tau_{n})$. By Talagrand's Inequality in the Noise Norm for a Diagonal Gaussian Reference Measure on a Hilbert Space §connected and Talagrand's Inequality in the Noise Norm for a Diagonal Gaussian Reference Measure on a Hilbert Space §talagrand, $\alpha_{i}\in\mathcal{P}^{a}_{\rho}$ and $W_{a}(\alpha_{i},\gamma_{c})^{2}\le2\kappa\,H(\alpha_{i}\,|\,\gamma_{c})\le2\kappa\,H(\sigma_{i}\,|\,\tau_{n})$. As $\alpha_{i}\in\mathcal{P}^{a}_{\rho}$, the pair $(\alpha_{i},\gamma_{c})$ is noise-connected by The Measures Noise-Connected to the Reference Measure §space, so by The Noise Wasserstein Distance is a Metric on the Measures Noise-Connected to the Reference Measure: Existence of Noise-Optimal Couplings, Comparison with the Quadratic Wasserstein Distance and Lower Semicontinuity §optimal there is a noise-optimal coupling $\chi_{i}\in\Pi^{a}(\alpha_{i},\gamma_{c})$, with $I^{a}(\chi_{i})=W_{a}(\alpha_{i},\gamma_{c})^{2}$.

Let $G_{n}=(Q_{n}\circ\pi_{1},Q_{n}\circ\pi_{2}):X\times X\to X\times X$, which is Borel by Borel Sets of a Hilbert Space with an Orthonormal Basis: Coordinates, Determination by Finite-Dimensional Projections, and Pairs §continuity, Borel Sets of a Hilbert Space with an Orthonormal Basis: Coordinates, Determination by Finite-Dimensional Projections, and Pairs §product-sigma and Borel Sets of a Hilbert Space with an Orthonormal Basis: Coordinates, Determination by Finite-Dimensional Projections, and Pairs §pairing, and let $\vartheta_{i}=(G_{n})_{\#}\chi_{i}\in\mathcal{P}(X\times X)$. Its first marginal is $(Q_{n})_{\#}\alpha_{i}=(Q_{n}\circ\Psi_{n})_{\#}(\tilde{\gamma}_{n}\otimes\sigma_{i})$; by (b) of Step 2, $Q_{n}\circ\Psi_{n}(u,w)=Q_{n}w$, and the second marginal of $\tilde{\gamma}_{n}\otimes\sigma_{i}$ is $\sigma_{i}$ (its value on $\mathbb{R}^{n}\times B$ is $\sigma_{i}(B)$ by Existence and Uniqueness of the Product Measure), so the first marginal of $\vartheta_{i}$ is $(Q_{n})_{\#}\sigma_{i}=(Q_{n}\circ Q_{n})_{\#}\mu_{i}=\sigma_{i}$ by (a) of Step 2. Its second marginal is $(Q_{n})_{\#}\gamma_{c}=\tau_{n}$. So $\vartheta_{i}\in\Pi(\sigma_{i},\tau_{n})$. If $(x,y)\in D_{a}$, then $Q_{n}y-Q_{n}x=Q_{n}(y-x)\in X^{a}$ with $n_{a}(Q_{n}(y-x))\le n_{a}(y-x)$ by (d) of Step 2, i.e. $G_{n}(x,y)\in D_{a}$ and $c_{a}(G_{n}(x,y))\le c_{a}(x,y)$. Since $\chi_{i}(D_{a})=1$ by Couplings of Finite Noise Cost and Their Noise Cost §finite, we get $\vartheta_{i}(D_{a})=\chi_{i}(G_{n}^{-1}(D_{a}))\ge\chi_{i}(D_{a})=1$, and by The Lebesgue Integral and Null Sets: Almost-Everywhere Comparison, Markov's Inequality, and Dominated Convergence Almost Everywhere §comparison,

$$\int c_{a}\,d\vartheta_{i}=\int c_{a}\circ G_{n}\,d\chi_{i}\le\int c_{a}\,d\chi_{i}=I^{a}(\chi_{i}).$$

Hence $\vartheta_{i}\in\Pi^{a}(\sigma_{i},\tau_{n})$ (Couplings of Finite Noise Cost and Their Noise Cost §couplings) and

$$I^{a}(\vartheta_{i})\le W_{a}(\alpha_{i},\gamma_{c})^{2}\le2\kappa\,H(\sigma_{i}\,|\,\tau_{n}).$$

Step 7 (A coupling of $\mu_{i}$ and $E_{n}(\tilde{\mu}_{i})$). Fix $i\in J$. The map $(\mathrm{id}_{X},Q_{n}):X\to X\times X$ is Borel by Borel Sets of a Hilbert Space with an Orthonormal Basis: Coordinates, Determination by Finite-Dimensional Projections, and Pairs §pairing, and $\beta_{i}=(\mathrm{id}_{X},Q_{n})_{\#}\mu_{i}$ has marginals $\mu_{i}$ and $(Q_{n})_{\#}\mu_{i}=\sigma_{i}$, so $\beta_{i}\in\Pi(\mu_{i},\sigma_{i})$. Since $\mu_{i},\sigma_{i},\tau_{n}\in\mathcal{P}_{2}(X)$ (Step 5) and $\vartheta_{i}\in\Pi(\sigma_{i},\tau_{n})$, Gluing Two Couplings on a Hilbert Space over a Common Middle Marginal, and the Triangle Inequalities for the Quadratic and Noise Costs §glued gives $\theta_{i}\in\mathcal{P}(X_{(3)})$, with $X_{(3)}$ and the Borel maps $q_{1},q_{2},q_{3}$ of that lemma, such that $(q_{1},q_{2})_{\#}\theta_{i}=\beta_{i}$ and $(q_{2},q_{3})_{\#}\theta_{i}=\vartheta_{i}$. Hence $(q_{1})_{\#}\theta_{i}=(\pi_{1})_{\#}\beta_{i}=\mu_{i}$ and $(q_{3})_{\#}\theta_{i}=(\pi_{2})_{\#}\vartheta_{i}=\tau_{n}$.

Let $\Gamma=\{(x,y)\in X\times X:y=Q_{n}x\}$, which is closed since $Q_{n}$ is continuous, hence Borel; $\beta_{i}(\Gamma)=\mu_{i}(X)=1$. Let $N_{i}\subseteq X_{(3)}$ be the union of $(q_{1},q_{2})^{-1}(X\times X\setminus\Gamma)$, $(q_{2},q_{3})^{-1}(X\times X\setminus D_{a})$ and $q_{3}^{-1}(X\setminus Z_{n})$. These sets are Borel with $\theta_{i}$-measures $1-\beta_{i}(\Gamma)=0$, $1-\vartheta_{i}(D_{a})=0$ and $1-\tau_{n}(Z_{n})=0$ (by (f) of Step 2), so $\theta_{i}(N_{i})=0$ by The Lebesgue Integral and Null Sets: Almost-Everywhere Comparison, Markov's Inequality, and Dominated Convergence Almost Everywhere §null-union. For $\omega\in X_{(3)}\setminus N_{i}$ we have $q_{2}(\omega)=Q_{n}q_{1}(\omega)$, $q_{3}(\omega)\in Z_{n}$, and $q_{3}(\omega)-Q_{n}q_{1}(\omega)=q_{3}(\omega)-q_{2}(\omega)\in X^{a}$.

Let $\omega_{i}=\theta_{i}\otimes\tilde{\mu}_{i}$ be the product measure on $\mathcal{B}(X_{(3)})\otimes\mathcal{B}(\mathbb{R}^{n})$ given by Existence and Uniqueness of the Product Measure, a probability measure. Define $\Theta:X_{(3)}\times\mathbb{R}^{n}\to\mathbb{R}^{n}\times X$ by $\Theta(\omega,v)=(v,q_{3}(\omega))$ and $\Phi:X_{(3)}\times\mathbb{R}^{n}\to X\times X$ by $\Phi(\omega,v)=(q_{1}(\omega),\Psi_{n}(v,q_{3}(\omega)))$. The map $\Theta$ is measurable from $\mathcal{B}(X_{(3)})\otimes\mathcal{B}(\mathbb{R}^{n})$ to $\mathcal{B}(\mathbb{R}^{n})\otimes\mathcal{B}(X)$: the sets whose preimage is measurable form a $\sigma$-algebra (preimages commute with complements and countable unions), and it contains every rectangle $F\times B$, whose preimage is the measurable rectangle $q_{3}^{-1}(B)\times F$, so it contains the $\sigma$-algebra these rectangles generate, which is the product $\sigma$-algebra. In the same way $(\omega,v)\mapsto q_{1}(\omega)$ is measurable, the preimage of $B$ being $q_{1}^{-1}(B)\times\mathbb{R}^{n}$. As $\Psi_{n}$ is measurable for $\mathcal{B}(\mathbb{R}^{n})\otimes\mathcal{B}(X)$ by The Gaussian-Tail Extension of a Probability Measure on the Rescaled Head §borel, $\Phi$ is measurable into $\mathcal{B}(X\times X)$ by Borel Sets of a Hilbert Space with an Orthonormal Basis: Coordinates, Determination by Finite-Dimensional Projections, and Pairs §pairing. Let $\varpi_{i}=\Phi_{\#}\omega_{i}\in\mathcal{P}(X\times X)$.

The first marginal of $\varpi_{i}$ assigns to $B\in\mathcal{B}(X)$ the value $\omega_{i}(q_{1}^{-1}(B)\times\mathbb{R}^{n})=\mu_{i}(B)$. The measure $\Theta_{\#}\omega_{i}$ on $\mathcal{B}(\mathbb{R}^{n})\otimes\mathcal{B}(X)$ assigns to $F\times B$ the value $\omega_{i}(q_{3}^{-1}(B)\times F)=\tau_{n}(B)\,\tilde{\mu}_{i}(F)$, so it equals $\tilde{\mu}_{i}\otimes\tau_{n}$ by the uniqueness in Existence and Uniqueness of the Product Measure; as $(\pi_{2})\circ\Phi=\Psi_{n}\circ\Theta$, the second marginal of $\varpi_{i}$ is $(\Psi_{n})_{\#}(\tilde{\mu}_{i}\otimes\tau_{n})=E_{n}(\tilde{\mu}_{i})$ by The Gaussian-Tail Extension of a Probability Measure on the Rescaled Head §extension. Thus $\varpi_{i}\in\Pi(\mu_{i},E_{n}(\tilde{\mu}_{i}))$.

For $(\omega,v)$ with $\omega\notin N_{i}$, (e) of Step 2, applied with $x=q_{1}(\omega)$, $w'=q_{3}(\omega)$ and $t=q_{3}(\omega)-q_{2}(\omega)$, gives $\Phi(\omega,v)\in D_{a}$ and

$$c_{a}(\Phi(\omega,v))=\lVert v-r_{n}(q_{1}(\omega))\rVert^{2}+c_{a}(q_{2}(\omega),q_{3}(\omega)).$$

The set $N_{i}\times\mathbb{R}^{n}$ has $\omega_{i}$-measure $\theta_{i}(N_{i})\,\tilde{\mu}_{i}(\mathbb{R}^{n})=0$. Hence $\varpi_{i}(D_{a})=\omega_{i}(\Phi^{-1}(D_{a}))=1$. Both summands on the right are measurable functions of $(\omega,v)$: the first, $(\omega,v)\mapsto\lVert v-r_{n}(q_{1}(\omega))\rVert^{2}$, is measurable by Pairs of Euclidean Points: Coordinate Projections, Pairings, the Product Measure on a Euclidean Space, Borel Norm Functions and Finite Sets §functions, applied on the measurable space $(X_{(3)}\times\mathbb{R}^{n},\mathcal{B}(X_{(3)})\otimes\mathcal{B}(\mathbb{R}^{n}))$ to the two maps $(\omega,v)\mapsto v$ and $(\omega,v)\mapsto r_{n}(q_{1}(\omega))$ into $\mathbb{R}^{n}$, which are measurable by the rectangle argument above (the preimage of $F\in\mathcal{B}(\mathbb{R}^{n})$ under the first is $X_{(3)}\times F$, and the second is the Borel map $r_{n}$ composed with the measurable map $(\omega,v)\mapsto q_{1}(\omega)$); the second is $c_{a}\circ(q_{2},q_{3})$ composed with the projection to $X_{(3)}$. By The Lebesgue Integral and Null Sets: Almost-Everywhere Comparison, Markov's Inequality, and Dominated Convergence Almost Everywhere §comparison and Tonelli and Fubini Theorems,

$$\int c_{a}\,d\varpi_{i}=\int c_{a}\circ\Phi\,d\omega_{i}=\ell_{i}+\int_{X_{(3)}}c_{a}\circ(q_{2},q_{3})\,d\theta_{i}=\ell_{i}+\int c_{a}\,d\vartheta_{i}=\ell_{i}+I^{a}(\vartheta_{i}),$$

where, integrating first in $\omega$ and using $(q_{1})_{\#}\theta_{i}=\mu_{i}$ and $(r_{n})_{\#}\mu_{i}=\tilde{\mu}_{i}$,

$$\ell_{i}=\int_{\mathbb{R}^{n}}\Bigl(\int_{X}\lVert v-r_{n}(x)\rVert^{2}\,\mu_{i}(dx)\Bigr)\tilde{\mu}_{i}(dv)=\int_{\mathbb{R}^{n}}\Bigl(\int_{\mathbb{R}^{n}}\lVert v-u\rVert^{2}\,\tilde{\mu}_{i}(du)\Bigr)\tilde{\mu}_{i}(dv)\in[0,\infty].$$

We bound $\ell_{i}$. If $i$ is small, then $\lVert v-u\rVert^{2}<\varepsilon$ for $u,v\in C_{i}$ and $\tilde{\mu}_{i}(\mathbb{R}^{n}\setminus C_{i})=0$, so two applications of The Lebesgue Integral and Null Sets: Almost-Everywhere Comparison, Markov's Inequality, and Dominated Convergence Almost Everywhere §comparison give $\ell_{i}\le\varepsilon$. If $i$ is large, then $\lVert v-u\rVert^{2}\le2\lVert u\rVert^{2}+2\lVert v\rVert^{2}$, so by Tonelli and Fubini Theorems and claim 3 of Image Measures, Measures with Densities, and Change of Variables with the density $p_{i}^{-1}\mathbf{1}_{C_{i}}$ of Step 5,

$$\ell_{i}\le4\int_{\mathbb{R}^{n}}\lVert u\rVert^{2}\,\tilde{\mu}_{i}(du)=\frac{4}{p_{i}}\int_{\mathbb{R}^{n}}\mathbf{1}_{C_{i}}(u)\lVert u\rVert^{2}\,\tilde{\mu}_{n}(du)\le\frac{4}{p_{i}}M_{2}(\tilde{\mu}_{n})<\infty .$$

In either case $\ell_{i}$ is finite, so $\varpi_{i}\in\Pi^{a}(\mu_{i},E_{n}(\tilde{\mu}_{i}))$ with $I^{a}(\varpi_{i})=\ell_{i}+I^{a}(\vartheta_{i})$.

Step 8 (Mixing the cells). Let $\varpi=\sum_{i\in J}p_{i}\varpi_{i}$, the set function $B\mapsto\sum_{i\in J}p_{i}\varpi_{i}(B)$ on $\mathcal{B}(X\times X)$; by Nonnegative Combinations of Two Finite Measures, and the Average of Two Couplings §combination, applied finitely many times (induction on the number of terms), it is a measure, $\varpi(X\times X)=\sum_{i\in J}p_{i}=1$ (as $\sum_{i\in[m]}p_{i}=1$ and $p_{i}=0$ for $i\notin J$, the preamble of The Partition Entropy of a Probability Measure Relative to Another over a Finite Measurable Partition), and $\int f\,d\varpi=\sum_{i\in J}p_{i}\int f\,d\varpi_{i}$ for every nonnegative Borel $f$. Its first marginal is $B\mapsto\sum_{i\in J}p_{i}\mu_{i}(B)=\mu(B)$ by Step 5. Its second marginal is $B\mapsto\sum_{i\in J}p_{i}(\tilde{\mu}_{i}\otimes\tau_{n})(\Psi_{n}^{-1}(B))$. The set function $\sum_{i\in J}p_{i}(\tilde{\mu}_{i}\otimes\tau_{n})$ on $\mathcal{B}(\mathbb{R}^{n})\otimes\mathcal{B}(X)$ is a measure by the same claim, and it assigns to a rectangle $F\times B$ the value $\sum_{i\in J}p_{i}\tilde{\mu}_{i}(F)\tau_{n}(B)=\tilde{\mu}_{n}(F)\tau_{n}(B)$ by Step 5; by the uniqueness in Existence and Uniqueness of the Product Measure it equals $\tilde{\mu}_{n}\otimes\tau_{n}$. Hence the second marginal of $\varpi$ is $(\Psi_{n})_{\#}(\tilde{\mu}_{n}\otimes\tau_{n})=E_{n}(\tilde{\mu}_{n})$ by The Gaussian-Tail Extension of a Probability Measure on the Rescaled Head §extension, and $\varpi\in\Pi(\mu,E_{n}(\tilde{\mu}_{n}))$. Moreover $\varpi(D_{a})=\sum_{i\in J}p_{i}\varpi_{i}(D_{a})=1$ and $\int c_{a}\,d\varpi=\sum_{i\in J}p_{i}I^{a}(\varpi_{i})<\infty$ by Step 7. Therefore $\varpi\in\Pi^{a}(\mu,E_{n}(\tilde{\mu}_{n}))$ by Couplings of Finite Noise Cost and Their Noise Cost §couplings, and

$$I^{a}(\varpi)=\sum_{i\in J}p_{i}\,\ell_{i}+\sum_{i\in J}p_{i}\,I^{a}(\vartheta_{i}).$$

Step 9 (The estimate; claim 2). Every $i\in J$ is small or large (Step 4 (iv)). The small indices contribute at most $\sum_{i\text{ small}}p_{i}\varepsilon\le\varepsilon$ to the first sum. For the large indices, the bound of Step 7 gives $p_{i}\ell_{i}\le4\int\mathbf{1}_{C_{i}}(u)\lVert u\rVert^{2}\,\tilde{\mu}_{n}(du)$, and the large cells are pairwise disjoint subsets of $U$, so their sum of indicators is at most $\mathbf{1}_{U}$; by Step 4 (i) the large indices contribute at most $4\int\mathbf{1}_{U}(u)\lVert u\rVert^{2}\,\tilde{\mu}_{n}(du)<4\varepsilon$. Hence $\sum_{i\in J}p_{i}\ell_{i}<5\varepsilon$. By Step 6 and $(*)$,

$$\sum_{i\in J}p_{i}\,I^{a}(\vartheta_{i})\le2\kappa\sum_{i\in J}p_{i}\,H(\sigma_{i}\,|\,\tau_{n})<2\kappa\bigl(H(\mu\,|\,\gamma_{c})-H(\tilde{\mu}_{n}\,|\,\tilde{\gamma}_{n})\bigr)+2\kappa\varepsilon .$$

By Step 1 and Step 8,

$$W_{a}\bigl(\mu,E_{n}(\tilde{\mu}_{n})\bigr)^{2}\le I^{a}(\varpi)<2\kappa\bigl(H(\mu\,|\,\gamma_{c})-H(\tilde{\mu}_{n}\,|\,\tilde{\gamma}_{n})\bigr)+(5+2\kappa)\varepsilon .$$

The left side and the quantity $\Delta_{n}=2\kappa(H(\mu\,|\,\gamma_{c})-H(\tilde{\mu}_{n}\,|\,\tilde{\gamma}_{n}))$ do not depend on $\varepsilon$, which was an arbitrary positive real number fixed before all choices of Step 4. If we had $W_{a}(\mu,E_{n}(\tilde{\mu}_{n}))^{2}>\Delta_{n}$, the choice $\varepsilon=(W_{a}(\mu,E_{n}(\tilde{\mu}_{n}))^{2}-\Delta_{n})/(5+2\kappa)$ would give a contradiction. Hence $W_{a}(\mu,E_{n}(\tilde{\mu}_{n}))^{2}\le\Delta_{n}$, which is claim 2.

Step 10 (Convergence; claim 3). Since $n$ was arbitrary in Steps 1 to 9, claim 2 holds for every $n\in\mathbb{N}$. By Gaussian-Tail Extensions: the Gaussian Reference Measure, Marginals, Densities, Relative Entropy, Relative Score and the Noise Wasserstein Distance §heads, $H(\tilde{\mu}_{n}\,|\,\tilde{\gamma}_{n})\to H(\mu\,|\,\gamma_{c})$ as $n\to\infty$, so $\Delta_{n}\to0$. Let $\xi>0$, and choose $n_{0}\in\mathbb{N}$ with $\Delta_{n}<\xi^{2}$ for $n\ge n_{0}$. For such $n$, claim 2 and the nonnegativity of $W_{a}$ (The Noise Wasserstein Distance §distance) give $0\le W_{a}(\mu,E_{n}(\tilde{\mu}_{n}))^{2}<\xi^{2}$, hence $0\le W_{a}(\mu,E_{n}(\tilde{\mu}_{n}))<\xi$. Thus $W_{a}(\mu,E_{n}(\tilde{\mu}_{n}))\to0$, which is claim 3.