Proof of Talagrand's Inequality for a Diagonal Gaussian Measure on Euclidean Space in the Cameron-Martin Cost

Each result cited is universally quantified over the data in its own statement.

Throughout, $d$, $c$, $\gamma_{c}$ and $\mu$ are as in the statement. Let $\rho_{c}$ be the diagonal Gaussian density with variances $c$, $Z_{c}$ its normalizing constant, $S_{c}$ the scaling map and $|x|_{c}^{2}=x\cdot S_{c}(x)=\sum_{k=1}^{d}x_{k}^{2}/c_{k}$ the weighted square of $x\in\mathbb{R}^{d}$; let $\mathrm{id}$ be the identity map of $\mathbb{R}^{d}$, and let $\lambda_{d}$ be Lebesgue measure on $\mathcal{B}(\mathbb{R}^{d})$. The entropy $\mathrm{Ent}$, the set $\mathcal{P}_{2}^{\mathrm{Ent}}(\mathbb{R}^{d})$, the score $\xi_{\nu}$, the tangent spaces $T_{\nu}$ and the spaces $L^{2}(\nu;\mathbb{R}^{d})$ with inner product $\langle\cdot,\cdot\rangle_{\nu}$ are those named in Relative Entropy and Relative Score with Respect to a Diagonal Gaussian Measure: Comparison with the Entropy, the Score and the Relative Free Energy of a Quadratic Potential. The proof runs McCann's tangent inequality at the source $\gamma_{c}$ along the Brenier map from $\gamma_{c}$ to $\mu$ for the ordinary Euclidean cost; the weighted cost appears only through the weighted squares in the comparison of relative entropy and entropy, so no rescaling of the variances is needed.

Step 1 ($\gamma_{c}$ relative to itself). The constant function $\mathbf{1}$ on $\mathbb{R}^{d}$ with value $1$ is Borel and nonnegative, and for every $B\in\mathcal{B}(\mathbb{R}^{d})$ one has $\int_{\mathbb{R}^{d}}\mathbf{1}_{B}\cdot1\,d\gamma_{c}=\int_{\mathbb{R}^{d}}\mathbf{1}_{B}\,d\gamma_{c}=\gamma_{c}(B)$ by The Integral of an Indicator Function is the Measure of the Set; so the constant $1$ is a density of $\gamma_{c}$ with respect to $\gamma_{c}$ in the sense of The Radon-Nikodym Theorem for a Finite Measure and a Sigma-Finite Measure, and Uniqueness of Densities. For the function $\phi$ of The Function $s\log s$: Continuity, Young's Inequality and Lower Bounds, with the Elementary Bounds for the Exponential and the Logarithm one has $\phi(1)=1\cdot\log1=0$, because $0=1-1^{-1}\le\log1\le1-1=0$ by The Function $s\log s$: Continuity, Young's Inequality and Lower Bounds, with the Elementary Bounds for the Exponential and the Logarithm §log. Hence $\phi\circ\mathbf{1}$ is the zero function, which is integrable with respect to $\gamma_{c}$ with integral $0\cdot\gamma_{c}(\mathbb{R}^{d})=0$ by claim 6(a) of Borel Measurability and Bounded Integration on a Metric Space ($\gamma_{c}$ being a Borel measure on $(\mathbb{R}^{d},d_{E})$ by Probability Measures on Euclidean Space and Random Vectors: Standing Notation §measures). By Relative Entropy of Probability Measures §relative-entropy, $\gamma_{c}$ has finite relative entropy with respect to $\gamma_{c}$ and $H(\gamma_{c}\,|\,\gamma_{c})=0$.

Step 2 (Second moments, entropies and absolute continuity). By The Entropy Inequality for Nonnegative Functions, and Quadratic Moments Bounded by the Relative Entropy with Respect to a Diagonal Gaussian Measure §second-moment, applied to $\mu$ and, by Step 1, to $\gamma_{c}$, both $\mu$ and $\gamma_{c}$ belong to $\mathcal{P}_{2}(\mathbb{R}^{d})$. By Relative Entropy and Relative Score with Respect to a Diagonal Gaussian Measure: Comparison with the Entropy, the Score and the Relative Free Energy of a Quadratic Potential §integrable, applied to each of them, the function $x\mapsto|x|_{c}^{2}$ is integrable (in particular Borel) with respect to $\mu$ and with respect to $\gamma_{c}$. By Relative Entropy and Relative Score with Respect to a Diagonal Gaussian Measure: Comparison with the Entropy, the Score and the Relative Free Energy of a Quadratic Potential §entropy, applied to $\mu$ and to $\gamma_{c}$, both have finite entropy, so both belong to $\mathcal{P}_{2}^{\mathrm{Ent}}(\mathbb{R}^{d})$, and, using Step 1 for the second identity,

$$H(\mu\,|\,\gamma_{c})=\mathrm{Ent}(\mu)+\tfrac12\int_{\mathbb{R}^{d}}|y|_{c}^{2}\,\mu(dy)+Z_{c},\qquad 0=\mathrm{Ent}(\gamma_{c})+\tfrac12\int_{\mathbb{R}^{d}}|x|_{c}^{2}\,\gamma_{c}(dx)+Z_{c}.\tag{1}$$

Since $\gamma_{c}$ has finite entropy, it is absolutely continuous by Basic Properties of the Entropy on the Wasserstein Space: Comparison with the Gaussian Relative Entropy, Lower Bound, Translation Invariance, Absolute Continuity, Closed Sublevel Sets and Lower Semicontinuity §absolutely-continuous.

Step 3 (The score of $\gamma_{c}$). We apply The Score of a Measure with a Positive Continuously Differentiable Density with $m=\rho_{c}$. By The Diagonal Gaussian Density on Euclidean Space: Regularity, Gradient, Normalization, Second Moments and Exponential Moments of Diagonal Quadratic Forms §regularity, $\rho_{c}$ is smooth and positive on $\mathbb{R}^{d}$, in particular of class $C^{1}$ there (Smooth Map on a Euclidean Open Set, with $k=1$); $\gamma_{c}$ is the measure with density $\rho_{c}$ with respect to $\lambda_{d}$ by Diagonal Gaussian Measures on Euclidean Space §measure, and it belongs to $\mathcal{P}_{2}(\mathbb{R}^{d})$ by Step 2. By The Diagonal Gaussian Density on Euclidean Space: Regularity, Gradient, Normalization, Second Moments and Exponential Moments of Diagonal Quadratic Forms §gradient, $D\rho_{c}(x)=-\rho_{c}(x)\,S_{c}(x)$, so the map $\eta$ of The Score of a Measure with a Positive Continuously Differentiable Density is $\eta(x)=\rho_{c}(x)^{-1}D\rho_{c}(x)=-S_{c}(x)$. Its squared norm $\lVert S_{c}\rVert^{2}$ has $\gamma_{c}$-integral at most $c_{\min}^{-2}M_{2}(\gamma_{c})<\infty$, where $c_{\min}$ is the least variance and $M_{2}(\gamma_{c})$ the second moment of $\gamma_{c}$, by the bound displayed in the preamble of Finite Fisher Information Relative to a Diagonal Gaussian Measure: the Ornstein-Uhlenbeck Functional, the Relative Score and the Relative Fisher Information, which holds for every member of $\mathcal{P}_{2}(\mathbb{R}^{d})$. Moreover $S_{c}\in T_{\gamma_{c}}$ by Relative Entropy and Relative Score with Respect to a Diagonal Gaussian Measure: Comparison with the Entropy, the Score and the Relative Free Energy of a Quadratic Potential §tangent, and $T_{\gamma_{c}}$ is a linear subspace of $L^{2}(\gamma_{c};\mathbb{R}^{d})$ by Basic Properties of the Tangent Space: Closed Subspace, the Identity Map Belongs to It, Second-Moment Limits, and Representation of Bounded Functionals on Gradients §closed, so $\eta=-S_{c}\in T_{\gamma_{c}}$. Hence $\gamma_{c}$ has finite Fisher information by The Score of a Measure with a Positive Continuously Differentiable Density §projection, and its score is

$$\xi_{\gamma_{c}}=-S_{c}$$

by The Score of a Measure with a Positive Continuously Differentiable Density §equality.

Step 4 (An optimal map). By Existence of an Optimal Coupling of Two Probability Measures with Finite Second Moment, read with $m=d$, there is an optimal coupling $\pi$ of $\gamma_{c}$ and $\mu$, both being in $\mathcal{P}_{2}(\mathbb{R}^{d})$ by Step 2. Since $\gamma_{c}$ is absolutely continuous (Step 2), Brenier's Theorem: Optimal Couplings out of an Absolutely Continuous Measure are Induced by a Unique Map §map, with $\gamma_{c}$ in place of $\mu$ and $\mu$ in place of $\nu$, gives a Borel map $T:\mathbb{R}^{d}\to\mathbb{R}^{d}$ with $\pi=(\mathrm{id},T)_{\#}\gamma_{c}$ and $T_{\#}\gamma_{c}=\mu$. Thus $T$ is an optimal map from $\gamma_{c}$ to $\mu$ in the sense of Optimal Transport Maps and Uniquely Mapped Pairs of Probability Measures §map. Its coordinate functions are written $T_{k}$, as in the statement.

Step 5 (The tangent inequality). By Steps 2 and 3, $\gamma_{c},\mu\in\mathcal{P}_{2}^{\mathrm{Ent}}(\mathbb{R}^{d})$ and $\gamma_{c}$ has finite Fisher information, and by Step 4, $T$ is an optimal map from $\gamma_{c}$ to $\mu$. So McCann's Tangent Inequality: the Entropy Lies Above its Tangent Along Optimal Maps §tangent, with $\gamma_{c}$ in place of $\mu$ and $\mu$ in place of $\nu$, gives $\mathrm{Ent}(\gamma_{c})+\langle\xi_{\gamma_{c}},T-\mathrm{id}\rangle_{\gamma_{c}}\le\mathrm{Ent}(\mu)$. By the preamble of McCann's Tangent Inequality: the Entropy Lies Above its Tangent Along Optimal Maps, the Borel maps $T$ and $\mathrm{id}$ are square-integrable random vectors on the probability space $(\mathbb{R}^{d},\mathcal{B}(\mathbb{R}^{d}),\gamma_{c})$, so their pointwise difference $T-\mathrm{id}$ is one too, by The Space of Square-Integrable Random Vectors §space read on that probability space as in Wasserstein Spaces, Random Vectors, Vector Fields and Symmetric Matrices in Every Dimension: Standing Notation §fields; that is, $x\mapsto T(x)-x$ is Borel and $\int_{\mathbb{R}^{d}}\lVert T(x)-x\rVert^{2}\,\gamma_{c}(dx)<\infty$. By Step 3, $S_{c}$ is Borel (preamble of Finite Fisher Information Relative to a Diagonal Gaussian Measure: the Ornstein-Uhlenbeck Functional, the Relative Score and the Relative Fisher Information) and $\int_{\mathbb{R}^{d}}\lVert S_{c}\rVert^{2}\,d\gamma_{c}<\infty$. Hence the function $f$ on $\mathbb{R}^{d}$ given by $f(x)=S_{c}(x)\cdot(T(x)-x)$ is Borel, and $|f(x)|\le\tfrac12\bigl(\lVert S_{c}(x)\rVert^{2}+\lVert T(x)-x\rVert^{2}\bigr)$ for every $x\in\mathbb{R}^{d}$, both by Pairs of Euclidean Points: Coordinate Projections, Pairings, the Product Measure on a Euclidean Space, Borel Norm Functions and Finite Sets §functions. So, by monotonicity, additivity and positive homogeneity of the integral of nonnegative measurable functions (Linearity and Monotonicity of the Lebesgue Integral §nonnegative),

$$\int_{\mathbb{R}^{d}}|f|\,d\gamma_{c}\le\tfrac12\int_{\mathbb{R}^{d}}\lVert S_{c}\rVert^{2}\,d\gamma_{c}+\tfrac12\int_{\mathbb{R}^{d}}\lVert T(x)-x\rVert^{2}\,\gamma_{c}(dx)<\infty,$$

and $f$ is integrable with respect to $\gamma_{c}$ by Integrable Function and the Lebesgue Integral. By Step 3, $\xi_{\gamma_{c}}$ is the class of $-S_{c}$, and $(-S_{c}(x))\cdot(T(x)-x)=-f(x)$ for every $x\in\mathbb{R}^{d}$; so the formula for the inner product of $L^{2}(\gamma_{c};\mathbb{R}^{d})$ in Wasserstein Spaces, Random Vectors, Vector Fields and Symmetric Matrices in Every Dimension: Standing Notation §fields, followed by Linearity and Monotonicity of the Lebesgue Integral §integrable (with $f$ in place of both $f$ and $g$, $a=-1$ and $b=0$), gives $\langle\xi_{\gamma_{c}},T-\mathrm{id}\rangle_{\gamma_{c}}=\int_{\mathbb{R}^{d}}(-f)\,d\gamma_{c}=-\int_{\mathbb{R}^{d}}S_{c}(x)\cdot(T(x)-x)\,\gamma_{c}(dx)$. Therefore

$$\mathrm{Ent}(\gamma_{c})-\int_{\mathbb{R}^{d}}S_{c}(x)\cdot\bigl(T(x)-x\bigr)\,\gamma_{c}(dx)\le\mathrm{Ent}(\mu).\tag{2}$$

Step 6 (The weighted square of the map). The function $y\mapsto|y|_{c}^{2}$ is Borel and integrable with respect to $\mu$ (Step 2; $\mu=T_{\#}\gamma_{c}$ by Step 4). By the change-of-variables formula of Probability Measures on Euclidean Space and Random Vectors: Standing Notation §pushforward, the function $x\mapsto|T(x)|_{c}^{2}$ is integrable with respect to $\gamma_{c}$ and

$$\int_{\mathbb{R}^{d}}|T(x)|_{c}^{2}\,\gamma_{c}(dx)=\int_{\mathbb{R}^{d}}|y|_{c}^{2}\,\mu(dy).\tag{3}$$

Step 7 (Conclusion). For every $x\in\mathbb{R}^{d}$ put $v=T(x)-x$. Expanding $(x_{k}+v_{k})^{2}=x_{k}^{2}+2x_{k}v_{k}+v_{k}^{2}$, dividing by $c_{k}$ and summing over $k\in\{1,\dots,d\}$ gives

$$|T(x)|_{c}^{2}=|x|_{c}^{2}+2\,S_{c}(x)\cdot\bigl(T(x)-x\bigr)+\sum_{k=1}^{d}\frac{\bigl(T_{k}(x)-x_{k}\bigr)^{2}}{c_{k}} .$$

Hence the nonnegative function $x\mapsto\sum_{k=1}^{d}c_{k}^{-1}(T_{k}(x)-x_{k})^{2}$ equals $|T(x)|_{c}^{2}-|x|_{c}^{2}-2\,S_{c}(x)\cdot(T(x)-x)$, a linear combination of three functions that are integrable with respect to $\gamma_{c}$ by Steps 6, 2 and 5; so it is Borel and integrable with respect to $\gamma_{c}$, and by Linearity and Monotonicity of the Lebesgue Integral §integrable

$$\tfrac12\int_{\mathbb{R}^{d}}\sum_{k=1}^{d}\frac{\bigl(T_{k}(x)-x_{k}\bigr)^{2}}{c_{k}}\,\gamma_{c}(dx) =\tfrac12\int_{\mathbb{R}^{d}}|T(x)|_{c}^{2}\,\gamma_{c}(dx)-\tfrac12\int_{\mathbb{R}^{d}}|x|_{c}^{2}\,\gamma_{c}(dx)-\int_{\mathbb{R}^{d}}S_{c}(x)\cdot\bigl(T(x)-x\bigr)\,\gamma_{c}(dx).$$

By (3), the first term on the right is $\tfrac12\int_{\mathbb{R}^{d}}|y|_{c}^{2}\,\mu(dy)$; by the second identity in (1), the second term is $\mathrm{Ent}(\gamma_{c})+Z_{c}$. So the right-hand side equals

$$\tfrac12\int_{\mathbb{R}^{d}}|y|_{c}^{2}\,\mu(dy)+Z_{c}+\Bigl(\mathrm{Ent}(\gamma_{c})-\int_{\mathbb{R}^{d}}S_{c}(x)\cdot\bigl(T(x)-x\bigr)\,\gamma_{c}(dx)\Bigr) \le\tfrac12\int_{\mathbb{R}^{d}}|y|_{c}^{2}\,\mu(dy)+Z_{c}+\mathrm{Ent}(\mu)=H(\mu\,|\,\gamma_{c}),$$

by (2) and then the first identity in (1). Multiplying by $2$ gives the inequality of the statement, for the Borel map $T$ of Step 4, which satisfies $T_{\#}\gamma_{c}=\mu$.