Proof of Moments of a Diagonal Gaussian Measure on a Hilbert Space: Coordinate Covariances, Finite Second Moment, and Exponential Moments of the Squared Norm

Each result cited below is universally quantified over the data in its own statement.

Throughout, $\mathcal{B}(\mathbb{R})$ is the Borel $\sigma$-algebra of the real line, which is the Borel $\sigma$-algebra of $(\mathbb{R},d_{\mathbb{R}})$ by claim 2 of Borel Measurability and Bounded Integration on a Metric Space. Hence, for a measurable space $(E,\mathcal{E})$, a function $f:E\to\mathbb{R}$ is measurable with respect to $\mathcal{E}$ and $\mathcal{B}(\mathbb{R})$, and in particular a function on $X$ or on $\mathbb{R}^{n}$ is Borel in the sense of Borel Probability Measures on a Real Hilbert Space with an Orthonormal Basis: Standing Notation §measures, exactly when $\{e\in E:f(e)>t\}\in\mathcal{E}$ for every real $t$, by claim 3 of Rational Intervals and Rays Generate the Borel Sigma-Algebra of the Real Line. For $n\in\mathbb{N}$ the $\sigma$-algebra $\mathcal{B}(\mathbb{R}^{n})$ of the metric space $(\mathbb{R}^{n},d_{E})$ is the one on which Lebesgue measure $\lambda_{n}$ and $\gamma_{c^{(n)}}$ are defined, by Euclidean Space and Lebesgue Measure: Standing Notation §borel. Write $\rho_{c^{(n)}}$ for the diagonal Gaussian density with variances $c^{(n)}$, a variance vector by Variance Sequences and Their Truncations §truncations; by Diagonal Gaussian Measures on Euclidean Space §measure, read with $n$ in place of $d$, $\gamma_{c^{(n)}}$ is the measure with density $\rho_{c^{(n)}}$ with respect to $\lambda_{n}$, and $\rho_{c^{(n)}}$ is positive and Borel by The Diagonal Gaussian Density on Euclidean Space: Regularity, Gradient, Normalization, Second Moments and Exponential Moments of Diagonal Quadratic Forms §regularity.

Step 1 (Three facts about $\exp$ and $\log$). (a) Let $(E,\mathcal{E})$ be a measurable space and $g:E\to\mathbb{R}$ measurable with respect to $\mathcal{E}$ and $\mathcal{B}(\mathbb{R})$; then so is $\exp\circ g$. Indeed, let $t$ be real. If $t\le0$, then $\{e\in E:\exp(g(e))>t\}=E$, because $\exp$ is positive by claim 2 of Basic Properties of the Exponential Function. If $t>0$, put $s=\log t$, so that $\exp(s)=t$ by The Natural Logarithm; since $\exp$ is strictly increasing by claim 4 of Basic Properties of the Exponential Function, $\exp(g(e))>\exp(s)$ holds exactly when $g(e)>s$, so $\{e\in E:\exp(g(e))>t\}=\{e\in E:g(e)>s\}\in\mathcal{E}$. The criterion of claim 3 of Rational Intervals and Rays Generate the Borel Sigma-Algebra of the Real Line gives the assertion.

(b) If $s\in\mathbb{R}$ and $(s_{m})_{m\in\mathbb{N}}$ is a sequence of real numbers converging to $s$, then $(\exp(s_{m}))_{m\in\mathbb{N}}$ converges to $\exp(s)$. Indeed, let $h$ be real with $|h|\le\tfrac12$. By The Function $s\log s$: Continuity, Young's Inequality and Lower Bounds, with the Elementary Bounds for the Exponential and the Logarithm §exp, $\exp(h)\ge1+h$ and $\exp(-h)\ge1-h\ge\tfrac12>0$; by claim 2 of Basic Properties of the Exponential Function, $\exp(h)=1/\exp(-h)\le1/(1-h)$, so

$$-|h|\le h\le\exp(h)-1\le\frac{h}{1-h}\le\frac{|h|}{1-h}\le2|h|,$$

that is, $|\exp(h)-1|\le2|h|$. By claim 1 of Basic Properties of the Exponential Function, $\exp(s_{m})-\exp(s)=\exp(s)\bigl(\exp(s_{m}-s)-1\bigr)$, so for every $m$ with $|s_{m}-s|\le\tfrac12$ we get $|\exp(s_{m})-\exp(s)|\le2\exp(s)|s_{m}-s|$, and the right side tends to $0$.

(c) If $v,\theta\in\mathbb{R}$ satisfy $0\le v\le\theta<1$, then $-\log(1-v)\le v/(1-\theta)$. Indeed $1-v\ge1-\theta>0$, and The Function $s\log s$: Continuity, Young's Inequality and Lower Bounds, with the Elementary Bounds for the Exponential and the Logarithm §log with $t=1-v$ gives $\log(1-v)\ge1-\frac{1}{1-v}=-\frac{v}{1-v}$; hence $-\log(1-v)\le\frac{v}{1-v}\le\frac{v}{1-\theta}$, the last inequality because $v\ge0$ and $0<1-\theta\le1-v$.

Step 2 (Realising $\gamma_{c}$ as a law). The standard normal distribution $N$ is a probability measure on $(\mathbb{R},\mathcal{B}(\mathbb{R}))$, so Existence of Independent and Identically Distributed Sequences provides a probability space $(\Omega,\mathcal{F},P)$ and a sequence $(\xi_{k})_{k\in\mathbb{N}}$ of random variables on it that is independent and identically distributed with common distribution $N$; thus $(\xi_{k})$ is an independent sequence of standard normal random variables. Let $Y_{n}$, $G$ and $Y$ be the maps and the set attached to this sequence in The Diagonal Gaussian Random Series in a Hilbert Space: Almost Sure Convergence, Finite-Dimensional Projections, and the Unique Measure with Diagonal Gaussian Projections. By The Diagonal Gaussian Random Series in a Hilbert Space: Almost Sure Convergence, Finite-Dimensional Projections, and the Unique Measure with Diagonal Gaussian Projections §convergence, $G\in\mathcal{F}$ and $P(G)=1$, so $P(\Omega\setminus G)=P(\Omega)-P(G)=0$ by Basic Properties of a Measure §differences, the measure $P$ being finite with $P(\Omega)=1$, and $\Omega\setminus G$ is a $P$-null set. By The Diagonal Gaussian Random Series in a Hilbert Space: Almost Sure Convergence, Finite-Dimensional Projections, and the Unique Measure with Diagonal Gaussian Projections §random-element, $Y$ is measurable with respect to $\mathcal{F}$ and $\mathcal{B}(X)$, and $\langle Y(\omega),e_{k}\rangle=\sqrt{c_{k}}\,\xi_{k}(\omega)$ for every $\omega\in G$ and $k\in\mathbb{N}$. By Diagonal Gaussian Measures on a Hilbert Space §measure, $\gamma_{c}$ is the unique $\mu\in\mathcal{P}(X)$ with $(p_{n})_{\#}\mu=\gamma_{c^{(n)}}$ for every $n$, and by The Diagonal Gaussian Random Series in a Hilbert Space: Almost Sure Convergence, Finite-Dimensional Projections, and the Unique Measure with Diagonal Gaussian Projections §unique this $\mu$ is the law $P_{Y}$ of $Y$; that is, $\gamma_{c}=P_{Y}$. By Random Element of a Metric Space and Its Law, $P_{Y}$ is the image measure of $P$ under $Y$. Hence claim 2 of Image Measures, Measures with Densities, and Change of Variables, applied to the measure space $(\Omega,\mathcal{F},P)$, the measurable space $(X,\mathcal{B}(X))$ and $T=Y$, gives: for every Borel $g:X\to\mathbb{R}$, $g$ is integrable with respect to $\gamma_{c}$ if and only if $g\circ Y$ is integrable with respect to $P$, and then

$$\int_{X}g\,d\gamma_{c}=\int_{\Omega}g\circ Y\,dP. \tag{$*$}$$

Step 3 (Claim 1). Fix $j,k\in\mathbb{N}$. The functions $x\mapsto x_{j}$ and $x\mapsto x_{k}$ are Borel by Borel Sets of a Hilbert Space with an Orthonormal Basis: Coordinates, Determination by Finite-Dimensional Projections, and Pairs §continuity, hence so is $x\mapsto x_{j}x_{k}$ by claim 3 of Arithmetic, Absolute Values, and Pointwise Limits of Measurable Real-Valued Functions, applied on $(X,\mathcal{B}(X))$. By claim 4 of Borel Measurability and Bounded Integration on a Metric Space, the functions $\omega\mapsto\langle Y(\omega),e_{k}\rangle$ and $\omega\mapsto\langle Y(\omega),e_{j}\rangle\langle Y(\omega),e_{k}\rangle$, the compositions of these two Borel functions with $Y$, are measurable with respect to $\mathcal{F}$. By Step 2, for every $\omega\in G$,

$$\langle Y(\omega),e_{k}\rangle=\sqrt{c_{k}}\,\xi_{k}(\omega),\qquad\langle Y(\omega),e_{j}\rangle\langle Y(\omega),e_{k}\rangle=\sqrt{c_{j}}\sqrt{c_{k}}\,\xi_{j}(\omega)\xi_{k}(\omega),$$

so both identities hold outside the null set $\Omega\setminus G$, that is, $P$-almost everywhere, in the sense of Measure Spaces and the Lebesgue Integral: Standing Notation §null.

By claim 1 of Moments and Stability of the Standard Normal Distribution, each $\xi_{k}$ and each $\xi_{k}^{2}$ is integrable, with $\int_{\Omega}\xi_{k}\,dP=0$ and $\int_{\Omega}\xi_{k}^{2}\,dP=1$, the expectations there being these integrals by Expectation, Variance, and Moments. By Linearity and Monotonicity of the Lebesgue Integral §integrable, $\sqrt{c_{k}}\,\xi_{k}$ is integrable with integral $0$. By The Lebesgue Integral and Null Sets: Almost-Everywhere Comparison, Markov's Inequality, and Dominated Convergence Almost Everywhere §comparison, the function $\omega\mapsto\langle Y(\omega),e_{k}\rangle$ is integrable with integral $0$; by $(*)$, $x\mapsto x_{k}$ is integrable with respect to $\gamma_{c}$ and $\int_{X}x_{k}\,\gamma_{c}(dx)=0$.

If $j=k$, then $\sqrt{c_{j}}\sqrt{c_{k}}=c_{k}$, since $\sqrt{c_{k}}$ is the nonnegative real number whose square is $c_{k}$ (Existence and Uniqueness of the Nonnegative Square Root); so the second identity above reads $\langle Y(\omega),e_{k}\rangle^{2}=c_{k}\,\xi_{k}(\omega)^{2}$ for $\omega\in G$. The function $c_{k}\xi_{k}^{2}$ is integrable with integral $c_{k}$ by Linearity and Monotonicity of the Lebesgue Integral §integrable, so by The Lebesgue Integral and Null Sets: Almost-Everywhere Comparison, Markov's Inequality, and Dominated Convergence Almost Everywhere §comparison and $(*)$, $x\mapsto x_{k}^{2}$ is integrable with respect to $\gamma_{c}$ and $\int_{X}x_{k}^{2}\,\gamma_{c}(dx)=c_{k}$.

If $j\ne k$, then $\xi_{j},\xi_{k}$ form a finite subfamily of the independent sequence $(\xi_{m})_{m\in\mathbb{N}}$, so they are independent by Independence of Events and of Random Variables; each has finite expectation $0$. By Expectation of a Product of Independent Random Variables, $\xi_{j}\xi_{k}$ has finite expectation, that is, it is integrable, and $\int_{\Omega}\xi_{j}\xi_{k}\,dP=0\cdot0=0$. By Linearity and Monotonicity of the Lebesgue Integral §integrable, $\sqrt{c_{j}}\sqrt{c_{k}}\,\xi_{j}\xi_{k}$ is integrable with integral $0$, so by The Lebesgue Integral and Null Sets: Almost-Everywhere Comparison, Markov's Inequality, and Dominated Convergence Almost Everywhere §comparison and $(*)$, $x\mapsto x_{j}x_{k}$ is integrable with respect to $\gamma_{c}$ and $\int_{X}x_{j}x_{k}\,\gamma_{c}(dx)=0$. This proves claim 1.

Step 4 (The truncated squared norms). For $n\in\mathbb{N}$ define $\varphi_{n}:\mathbb{R}^{n}\to\mathbb{R}$ by $\varphi_{n}(y)=\lVert y\rVert^{2}$ and $f_{n}:X\to\mathbb{R}$ by $f_{n}(x)=\varphi_{n}(p_{n}(x))$. By Euclidean Norm on $\mathbb{R}^n$ and Existence and Uniqueness of the Nonnegative Square Root, $\varphi_{n}(y)=\sum_{i=1}^{n}y_{i}^{2}$. By Borel Sets of a Hilbert Space with an Orthonormal Basis: Coordinates, Determination by Finite-Dimensional Projections, and Pairs §continuity, $|p_{n}^{*}(y)|=\lVert y\rVert$ and $|P_{n}x|=\lVert p_{n}(x)\rVert$, so

$$\varphi_{n}(y)=|p_{n}^{*}(y)|^{2},\qquad f_{n}(x)=|P_{n}x|^{2}=|x|^{2}-|Q_{n}x|^{2}\qquad(y\in\mathbb{R}^{n},\ x\in X),$$

the last equality again by Borel Sets of a Hilbert Space with an Orthonormal Basis: Coordinates, Determination by Finite-Dimensional Projections, and Pairs §continuity. The function $x\mapsto|x|^{2}$ is Borel on $X$, as recorded in the preamble of The Second Moment of a Borel Probability Measure on a Hilbert Space and the Probability Measures with Finite Second Moment, and $p_{n}^{*}$, $p_{n}$ are Borel by Borel Sets of a Hilbert Space with an Orthonormal Basis: Coordinates, Determination by Finite-Dimensional Projections, and Pairs §continuity; so $\varphi_{n}$ is Borel on $\mathbb{R}^{n}$ and $f_{n}=\varphi_{n}\circ p_{n}$ is Borel on $X$, by claim 4 of Borel Measurability and Bounded Integration on a Metric Space.

By Borel Probability Measures on a Real Hilbert Space with an Orthonormal Basis: Standing Notation §coordinates, $(X_{n})_{n\in\mathbb{N}}$ is an exhausting sequence for $X$ and $P_{n}$, $Q_{n}$ are the orthogonal projection onto $X_{n}$ and $x\mapsto x-P_{n}x$, which are the maps so named in Exhausting Sequences of Finite-Dimensional Subspaces in a Separable Real Hilbert Space, and Their Projections for this exhausting sequence. Fix $x\in X$. By Exhausting Sequences of Finite-Dimensional Subspaces in a Separable Real Hilbert Space, and Their Projections §projections, $|Q_{n+1}x|\le|Q_{n}x|$, so $|Q_{n+1}x|^{2}\le|Q_{n}x|^{2}$ and $0\le f_{n}(x)\le f_{n+1}(x)$ for every $n$. By Exhausting Sequences of Finite-Dimensional Subspaces in a Separable Real Hilbert Space, and Their Projections §tail, $Q_{n}x$ converges to $0_{X}$ in $(X,d)$, that is, $|Q_{n}x|=d(Q_{n}x,0_{X})\to0$; hence $|Q_{n}x|^{2}\to0$ and $f_{n}(x)\to|x|^{2}$. A nondecreasing sequence of real numbers converging to $|x|^{2}$ has least upper bound $|x|^{2}$, so

$$\sup_{n\in\mathbb{N}}f_{n}(x)=|x|^{2}\qquad(x\in X).$$

Step 5 (Integrals of functions of the first $n$ coordinates). Let $n\in\mathbb{N}$ and let $\psi:\mathbb{R}^{n}\to[0,\infty)$ be Borel. The map $p_{n}$ is Borel by Borel Sets of a Hilbert Space with an Orthonormal Basis: Coordinates, Determination by Finite-Dimensional Projections, and Pairs §continuity, $(p_{n})_{\#}\gamma_{c}$ is the image measure of $\gamma_{c}$ under $p_{n}$ by Borel Probability Measures on a Real Hilbert Space with an Orthonormal Basis: Standing Notation §pushforward, and $(p_{n})_{\#}\gamma_{c}=\gamma_{c^{(n)}}$ by Diagonal Gaussian Measures on a Hilbert Space §measure. Hence claim 2 of Image Measures, Measures with Densities, and Change of Variables, applied to $(X,\mathcal{B}(X),\gamma_{c})$, $(\mathbb{R}^{n},\mathcal{B}(\mathbb{R}^{n}))$ and $T=p_{n}$, followed by claim 3 of Image Measures, Measures with Densities, and Change of Variables, applied to $(\mathbb{R}^{n},\mathcal{B}(\mathbb{R}^{n}),\lambda_{n})$ and the density $\rho_{c^{(n)}}$, gives, in $[0,\infty]$,

$$\int_{X}\psi(p_{n}(x))\,\gamma_{c}(dx)=\int_{\mathbb{R}^{n}}\psi\,d\gamma_{c^{(n)}}=\int_{\mathbb{R}^{n}}\psi\,\rho_{c^{(n)}}\,d\lambda_{n}. \tag{$**$}$$

If moreover $\psi\rho_{c^{(n)}}$ is integrable with respect to $\lambda_{n}$, then, its negative part being $0$, whose integral is $0$ by The Lebesgue Integral and Null Sets: Almost-Everywhere Comparison, Markov's Inequality, and Dominated Convergence Almost Everywhere §null-integral (with the empty null set), its integral as an integrable function equals the integral in $(**)$, by Integrable Function and the Lebesgue Integral.

Step 6 (Claim 2). Fix $n\in\mathbb{N}$. By The Diagonal Gaussian Density on Euclidean Space: Regularity, Gradient, Normalization, Second Moments and Exponential Moments of Diagonal Quadratic Forms §moments, read with $n$ in place of $d$ and the variance vector $c^{(n)}$, each $y\mapsto y_{i}^{2}\rho_{c^{(n)}}(y)$ ($i\le n$) is integrable with respect to $\lambda_{n}$, so their sum $\varphi_{n}\rho_{c^{(n)}}$ is integrable by Linearity and Monotonicity of the Lebesgue Integral §integrable, with integral $\sum_{k=1}^{n}c_{k}$. Step 5 with $\psi=\varphi_{n}$ gives

$$\int_{X}f_{n}\,d\gamma_{c}=\sum_{k=1}^{n}c_{k}.$$

By Step 4 the functions $f_{n}$ are Borel and nonnegative, $f_{n}(x)\le f_{n+1}(x)$ for all $x$ and $n$, and $\sup_{n}f_{n}(x)=|x|^{2}$. By Monotone Convergence Theorem on $(X,\mathcal{B}(X),\gamma_{c})$, together with The Second Moment of a Borel Probability Measure on a Hilbert Space and the Probability Measures with Finite Second Moment §moment,

$$M_{2}(\gamma_{c})=\int_{X}|x|^{2}\,\gamma_{c}(dx)=\sup_{n\in\mathbb{N}}\sum_{k=1}^{n}c_{k}=\sum_{k=1}^{\infty}c_{k},$$

where the last equality holds because the partial sums are nondecreasing (each $c_{k}$ is positive) and converge to $\sum_{k=1}^{\infty}c_{k}$ by Variance Sequences and Their Truncations §variances. Since $\gamma_{c}\in\mathcal{P}(X)$ by Diagonal Gaussian Measures on a Hilbert Space §measure and $M_{2}(\gamma_{c})<\infty$, we get $\gamma_{c}\in\mathcal{P}_{2}(X)$ by The Second Moment of a Borel Probability Measure on a Hilbert Space and the Probability Measures with Finite Second Moment §space. This proves claim 2.

Step 7 (Claim 3). Let $\alpha,\theta$ be as in claim 3, and put $L=\frac{\alpha}{1-\theta}\sum_{k=1}^{\infty}c_{k}$, a real number since $\theta<1$. For each $k$, $v_{k}=2\alpha c_{k}$ satisfies $0\le v_{k}\le\theta<1$, since $\alpha\ge0$ and $c_{k}>0$.

Measurability. The function $x\mapsto\alpha|x|^{2}$ is Borel on $X$ (Step 4 and claim 2 of Arithmetic, Absolute Values, and Pointwise Limits of Measurable Real-Valued Functions), so $u(x)=\exp(\alpha|x|^{2})$ defines a Borel function $u:X\to\mathbb{R}$ by Step 1(a), which is positive by claim 2 of Basic Properties of the Exponential Function. For $n\in\mathbb{N}$, $\psi_{n}(y)=\exp(\alpha\varphi_{n}(y))$ defines a positive Borel function on $\mathbb{R}^{n}$ in the same way, and $u_{n}(x)=\psi_{n}(p_{n}(x))=\exp(\alpha f_{n}(x))$ a positive Borel function on $X$, by claim 4 of Borel Measurability and Bounded Integration on a Metric Space.

Monotone approximation. Fix $x\in X$. By Step 4 and $\alpha\ge0$, $\alpha f_{n}(x)\le\alpha f_{n+1}(x)$, so $u_{n}(x)\le u_{n+1}(x)$ because $\exp$ is increasing (claim 4 of Basic Properties of the Exponential Function); and $\alpha f_{n}(x)\to\alpha|x|^{2}$, so $u_{n}(x)\to u(x)$ by Step 1(b). Hence $\sup_{n}u_{n}(x)=u(x)$.

The approximating integrals. Fix $n\in\mathbb{N}$. By Step 4, $\psi_{n}(y)=\exp\bigl(\sum_{i=1}^{n}\alpha y_{i}^{2}\bigr)$. Since $2\alpha c_{i}=v_{i}<1$ for $i\le n$, The Diagonal Gaussian Density on Euclidean Space: Regularity, Gradient, Normalization, Second Moments and Exponential Moments of Diagonal Quadratic Forms §exponential, read with $n$ in place of $d$, the variance vector $c^{(n)}$ and $t=(\alpha,\dots,\alpha)\in\mathbb{R}^{n}$, shows that $\psi_{n}\rho_{c^{(n)}}$ is integrable with respect to $\lambda_{n}$ with integral $\exp\bigl(-\tfrac12\sum_{i=1}^{n}\log(1-2\alpha c_{i})\bigr)$. By Step 1(c) with $v=v_{i}$, $-\tfrac12\log(1-2\alpha c_{i})\le\tfrac12\cdot\frac{2\alpha c_{i}}{1-\theta}=\frac{\alpha c_{i}}{1-\theta}$ for each $i\le n$, so

$$-\frac12\sum_{i=1}^{n}\log(1-2\alpha c_{i})\le\frac{\alpha}{1-\theta}\sum_{i=1}^{n}c_{i}\le L,$$

the last step because $\frac{\alpha}{1-\theta}\ge0$ and, the $c_{k}$ being positive, each partial sum is at most $\sum_{k=1}^{\infty}c_{k}$. As $\exp$ is increasing, Step 5 with $\psi=\psi_{n}$ yields

$$\int_{X}u_{n}\,d\gamma_{c}=\exp\Bigl(-\frac12\sum_{i=1}^{n}\log(1-2\alpha c_{i})\Bigr)\le\exp(L).$$

Conclusion. By Monotone Convergence Theorem on $(X,\mathcal{B}(X),\gamma_{c})$ and the monotone approximation above,

$$\int_{X}u\,d\gamma_{c}=\sup_{n\in\mathbb{N}}\int_{X}u_{n}\,d\gamma_{c}\le\exp(L)<\infty .$$

Since $u$ is Borel, real-valued and $|u|=u$, it is integrable with respect to $\gamma_{c}$ by the criterion of Measure Spaces and the Lebesgue Integral: Standing Notation §integral, and its integral as an integrable function equals the integral just bounded, its negative part being $0$ (Integrable Function and the Lebesgue Integral), with integral $0$ by The Lebesgue Integral and Null Sets: Almost-Everywhere Comparison, Markov's Inequality, and Dominated Convergence Almost Everywhere §null-integral (with the empty null set). Thus

$$\int_{X}\exp\bigl(\alpha|x|^{2}\bigr)\,\gamma_{c}(dx)\le\exp\Bigl(\frac{\alpha}{1-\theta}\sum_{k=1}^{\infty}c_{k}\Bigr),$$

which proves claim 3.