Proof of The Paley-Wiener Functional of a Cameron-Martin Vector: Borel Measurability, Almost Everywhere and Mean-Square Convergence, Mean Zero and Variance the Cameron-Martin Square

Each result cited below is universally quantified over the data in its own statement.

Throughout, for $h\in X$ let $D_{h}$ denote the set of those $x\in X$ for which the real sequence $(\ell_{h,n}(x))_{n\in\mathbb{N}}$ converges. A real-valued function on $X$ is Borel when it is measurable with respect to $\mathcal{B}(X)$ and the Borel $\sigma$-algebra of $(\mathbb{R},d_{\mathbb{R}})$, as in Borel Probability Measures on a Real Hilbert Space with an Orthonormal Basis: Standing Notation §measures; by claim 2 of Borel Measurability and Bounded Integration on a Metric Space the latter is the Borel $\sigma$-algebra $\mathcal{B}(\mathbb{R})$ of the real line, so a Borel function on $X$ is a measurable real-valued function on the measurable space $(X,\mathcal{B}(X))$ in the sense of Arithmetic, Absolute Values, and Pointwise Limits of Measurable Real-Valued Functions, and conversely.

Step 1 (the partial sums are Borel). Fix $h\in X$ and $n\in\mathbb{N}$. By the definition of the partial sums, $\ell_{h,n}(x)=\sum_{k=1}^{n}(h_{k}/c_{k})\,x_{k}$ for every $x\in X$, where each $c_{k}$ is positive by Variance Sequences and Their Truncations §variances; thus $\ell_{h,n}$ is the linear combination, with the real coefficients $h_{1}/c_{1},\dots,h_{n}/c_{n}$, of the coordinate functions $x\mapsto x_{1},\dots,x\mapsto x_{n}$. Each coordinate function is Lipschitz, hence Borel, by Borel Sets of a Hilbert Space with an Orthonormal Basis: Coordinates, Determination by Finite-Dimensional Projections, and Pairs §continuity, and therefore $\ell_{h,n}$ is Borel by claim 2 of Arithmetic, Absolute Values, and Pointwise Limits of Measurable Real-Valued Functions.

Step 2 (the convergence set is Borel). Fix $h\in X$. For $r,m,n\in\mathbb{N}$ put $u_{m,n}=|\ell_{h,m}-\ell_{h,n}|$, a Borel function by Step 1 and claims 2 and 4 of Arithmetic, Absolute Values, and Pointwise Limits of Measurable Real-Valued Functions, and

$$A_{r,m,n}=\{x\in X:u_{m,n}(x)\le 1/r\}=X\setminus u_{m,n}^{-1}\bigl((1/r,\infty)\bigr).$$

The open interval $(1/r,\infty)$ is an open subset of $(\mathbb{R},d_{\mathbb{R}})$, hence a Borel set by claim 1 of Borel Measurability and Bounded Integration on a Metric Space, so its preimage under the Borel function $u_{m,n}$ lies in $\mathcal{B}(X)$, and so does its complement $A_{r,m,n}$, because $\mathcal{B}(X)$ is a $\sigma$-algebra by Borel Sigma-Algebra of a Metric Space. We claim

$$D_{h}=\bigcap_{r\in\mathbb{N}}\ \bigcup_{N\in\mathbb{N}}\ \bigcap_{m\ge N}\ \bigcap_{n\ge N}A_{r,m,n}.$$

Let $x\in X$ and write $a_{n}=\ell_{h,n}(x)$. The point $x$ lies in the right-hand side exactly when for every $r\in\mathbb{N}$ there is $N\in\mathbb{N}$ with $|a_{m}-a_{n}|\le1/r$ for all $m,n\ge N$. If $(a_{n})$ converges to a real number $a$, then for $r\in\mathbb{N}$ there is $N$ with $|a_{n}-a|<1/(2r)$ for $n\ge N$, whence $|a_{m}-a_{n}|<1/r$ for $m,n\ge N$; so $x$ lies in the right-hand side. Conversely, if $x$ lies in the right-hand side, then for every positive real $\varepsilon$ choose $r\in\mathbb{N}$ with $1/r<\varepsilon$ and the corresponding $N$, so that $|a_{m}-a_{n}|<\varepsilon$ for $m,n\ge N$; thus $(a_{n})$ is a Cauchy sequence and converges by Every Cauchy Sequence of Real Numbers Converges, that is, $x\in D_{h}$. The right-hand side is formed from members of $\mathcal{B}(X)$ by countable intersections and countable unions, so $D_{h}\in\mathcal{B}(X)$.

Step 3 (the functional is Borel; claim 1). Fix $h\in X$. The indicator $\mathbf{1}_{D_{h}}$ is Borel by Step 2 and claim 1 of Arithmetic, Absolute Values, and Pointwise Limits of Measurable Real-Valued Functions, so each product $f_{n}=\mathbf{1}_{D_{h}}\,\ell_{h,n}$ is Borel by Step 1 and claim 3 of that lemma. For every $x\in X$ the sequence $(f_{n}(x))_{n\in\mathbb{N}}$ converges to $\ell_{h}(x)$: if $x\in D_{h}$ then $f_{n}(x)=\ell_{h,n}(x)$ for every $n$ and the limit is $\ell_{h}(x)$ by the definition of the Paley-Wiener functional; if $x\notin D_{h}$ then $f_{n}(x)=0$ for every $n$ and $\ell_{h}(x)=0$ by the same definition. Hence $\ell_{h}$ is Borel by claim 5 of Arithmetic, Absolute Values, and Pointwise Limits of Measurable Real-Valued Functions. Together with Steps 1 and 2 this proves claim 1.

Step 4 (basis vectors; claim 2). Fix $k\in\mathbb{N}$ and $x\in X$, and take $h=e_{k}$. Since $(e_{j})_{j\in\mathbb{N}}$ is an orthonormal basis, it is an orthonormal sequence, so the coordinates of $e_{k}$ are $\langle e_{k},e_{j}\rangle=0$ for $j\ne k$ and $\langle e_{k},e_{k}\rangle=|e_{k}|^{2}=1$. Hence $\ell_{e_{k},n}(x)=0$ for $n<k$ and $\ell_{e_{k},n}(x)=x_{k}/c_{k}$ for every $n\ge k$. The real sequence $(\ell_{e_{k},n}(x))_{n\in\mathbb{N}}$ is therefore constant from the index $k$ on, so it converges to $x_{k}/c_{k}$; thus $x\in D_{e_{k}}$ and $\ell_{e_{k}}(x)=x_{k}/c_{k}$ by The Paley-Wiener Functional of a Vector Relative to a Variance Sequence §functional.

Step 5 (a random series realising $\gamma_{c}$). The standard normal distribution $N$ of Standard Normal Distribution is a probability measure on $(\mathbb{R},\mathcal{B}(\mathbb{R}))$, so by Existence of Independent and Identically Distributed Sequences there are a probability space $(\Omega,\mathcal{F},P)$ and an independent and identically distributed sequence $(\xi_{k})_{k\in\mathbb{N}}$ of random variables on it with common distribution $N$; that is, $(\xi_{k})$ is an independent sequence of standard normal random variables. Let $G$ and $Y:\Omega\to X$ be the event and the map attached to these data in The Diagonal Gaussian Random Series in a Hilbert Space: Almost Sure Convergence, Finite-Dimensional Projections, and the Unique Measure with Diagonal Gaussian Projections, with $\sqrt{c_{k}}$ the nonnegative square root of $c_{k}$ (Existence and Uniqueness of the Nonnegative Square Root), which satisfies $(\sqrt{c_{k}})^{2}=c_{k}$ and is positive because $c_{k}>0$ (if it were $0$, then $c_{k}=0$). By The Diagonal Gaussian Random Series in a Hilbert Space: Almost Sure Convergence, Finite-Dimensional Projections, and the Unique Measure with Diagonal Gaussian Projections §convergence, $G\in\mathcal{F}$ and $P(G)=1$; by The Diagonal Gaussian Random Series in a Hilbert Space: Almost Sure Convergence, Finite-Dimensional Projections, and the Unique Measure with Diagonal Gaussian Projections §random-element, $Y$ is a random element of $(X,d)$, that is, measurable with respect to $\mathcal{F}$ and $\mathcal{B}(X)$, and $\langle Y(\omega),e_{k}\rangle=\sqrt{c_{k}}\,\xi_{k}(\omega)$ for all $\omega\in G$ and $k\in\mathbb{N}$. By Diagonal Gaussian Measures on a Hilbert Space §measure, $\gamma_{c}$ is the unique $\mu\in\mathcal{P}(X)$ with $(p_{n})_{\#}\mu=\gamma_{c^{(n)}}$ for every $n$, and by The Diagonal Gaussian Random Series in a Hilbert Space: Almost Sure Convergence, Finite-Dimensional Projections, and the Unique Measure with Diagonal Gaussian Projections §unique this $\mu$ is the law $P_{Y}$ of $Y$. Thus $\gamma_{c}=P_{Y}$, the image measure of $P$ under $Y$ on $(X,\mathcal{B}(X))$ given by $\gamma_{c}(B)=P(Y^{-1}(B))$ for $B\in\mathcal{B}(X)$, as in Random Element of a Metric Space and Its Law. Consequently, by claim 2 of Image Measures, Measures with Densities, and Change of Variables (with $T=Y$ and $\mu_{T}=P_{Y}=\gamma_{c}$), for every Borel $g:X\to\mathbb{R}$: if $g\ge0$ then $\int_{X}g\,d\gamma_{c}=\int_{\Omega}g\circ Y\,dP$ in $[0,\infty]$; and $g$ is integrable with respect to $\gamma_{c}$ if and only if $g\circ Y$ is integrable with respect to $P$, in which case $\int_{X}g\,d\gamma_{c}=\int_{\Omega}g\circ Y\,dP$. For every Borel $g:X\to\mathbb{R}$ the function $g\circ Y$ is measurable with respect to $\mathcal{F}$ and $\mathcal{B}(\mathbb{R})$ by claims 4 and 2 of Borel Measurability and Bounded Integration on a Metric Space, that is, a random variable.

Step 6 (the scaled coordinates satisfy Kolmogorov's hypotheses). Fix $h\in H_{c}$ and put $b_{k}=h_{k}/\sqrt{c_{k}}$ and $\eta_{k}=b_{k}\,\xi_{k}$ for $k\in\mathbb{N}$. Each $\eta_{k}$ is a random variable and is square-integrable, since $\xi_{k}$ is square-integrable by claim 1 of Moments and Stability of the Standard Normal Distribution and scalar multiples of (square-integrable) random variables are again such by Square-Integrable Random Variables and the Mean-Square Inner Product. The sequence $(\eta_{k})$ is independent: let $k_{1},\dots,k_{r}$ be distinct natural numbers and $B_{1},\dots,B_{r}$ Borel sets; the map $t\mapsto b_{k_{i}}t$ on $\mathbb{R}$ is continuous for $d_{\mathbb{R}}$, hence measurable with respect to $\mathcal{B}(\mathbb{R})$ by claims 3 and 2 of Borel Measurability and Bounded Integration on a Metric Space, so $B_{i}'=\{t\in\mathbb{R}:b_{k_{i}}t\in B_{i}\}$ is a Borel set and $\{\eta_{k_{i}}\in B_{i}\}=\{\xi_{k_{i}}\in B_{i}'\}$; the events $\{\xi_{k_{1}}\in B_{1}'\},\dots,\{\xi_{k_{r}}\in B_{r}'\}$ are independent because $(\xi_{k})$ is an independent sequence, and this is exactly the independence of $\eta_{k_{1}},\dots,\eta_{k_{r}}$ in the sense of Independence of Events and of Random Variables. By claim 1 of Moments and Stability of the Standard Normal Distribution, $\xi_{k}$ and $\xi_{k}^{2}$ are integrable with $\mathbb{E}[\xi_{k}]=0$ and $\mathbb{E}[\xi_{k}^{2}]=1$; since $\eta_{k}^{2}=b_{k}^{2}\,\xi_{k}^{2}$ pointwise, Linearity and Monotonicity of the Lebesgue Integral §integrable gives

$$\mathbb{E}[\eta_{k}]=b_{k}\,\mathbb{E}[\xi_{k}]=0,\qquad\mathbb{E}[\eta_{k}^{2}]=b_{k}^{2}\,\mathbb{E}[\xi_{k}^{2}]=\frac{h_{k}^{2}}{c_{k}},$$

using $(\sqrt{c_{k}})^{2}=c_{k}$; for the nonnegative integrable functions $\xi_{k}^{2}$ and $\eta_{k}^{2}$ the two readings of the expectation agree, since the negative part is $0$, with integral $0$ by The Lebesgue Integral and Null Sets: Almost-Everywhere Comparison, Markov's Inequality, and Dominated Convergence Almost Everywhere §null-integral. Since $h\in H_{c}$, the series $\sum_{k=1}^{\infty}h_{k}^{2}/c_{k}$ converges by The Cameron-Martin Space of a Diagonal Gaussian Measure on a Hilbert Space §space, with sum $|h|_{c}^{2}$ by The Cameron-Martin Space of a Diagonal Gaussian Measure on a Hilbert Space §square. So $(\eta_{k})$ satisfies the hypotheses of Kolmogorov's Maximal Inequality and the Convergence of Series of Independent Square-Integrable Random Variables; let $T_{0}=0$, $T_{n}=\sum_{k=1}^{n}\eta_{k}$, the set $C$ and the random variable $T$ be as there. By Kolmogorov's Maximal Inequality and the Convergence of Series of Independent Square-Integrable Random Variables §almost-sure, $C\in\mathcal{F}$ and $P(C)=1$; by Kolmogorov's Maximal Inequality and the Convergence of Series of Independent Square-Integrable Random Variables §mean-square, $T$ is square-integrable and, for every $n$ that is zero or a natural number,

$$\mathbb{E}\bigl[(T-T_{n})^{2}\bigr]=\sum_{k=n+1}^{\infty}\frac{h_{k}^{2}}{c_{k}},\qquad\mathbb{E}[T]=0,\qquad\mathbb{E}[T^{2}]=\sum_{k=1}^{\infty}\frac{h_{k}^{2}}{c_{k}}=|h|_{c}^{2},$$

and $\mathbb{E}[(T-T_{n})^{2}]\to0$ as $n\to\infty$. Each $T_{n}$ is a square-integrable random variable, as a finite sum of such, by Square-Integrable Random Variables and the Mean-Square Inner Product.

Step 7 (the functional along $Y$). Keep $h\in H_{c}$. For $\omega\in G$ and $n\in\mathbb{N}$, the coordinates of $Y(\omega)$ are $\sqrt{c_{k}}\,\xi_{k}(\omega)$ by Step 5, so

$$\ell_{h,n}(Y(\omega))=\sum_{k=1}^{n}\frac{h_{k}\sqrt{c_{k}}\,\xi_{k}(\omega)}{c_{k}}=\sum_{k=1}^{n}\frac{h_{k}}{\sqrt{c_{k}}}\,\xi_{k}(\omega)=T_{n}(\omega).$$

Let $E=G\cap C\in\mathcal{F}$. The events $\Omega\setminus G$ and $\Omega\setminus C$ have probability $1-1=0$ by claim 3 of Basic Properties of a Measure, so by countable subadditivity (claim 4 there, applied to the sequence $\Omega\setminus G,\Omega\setminus C,\emptyset,\emptyset,\dots$) the event $\Omega\setminus E=(\Omega\setminus G)\cup(\Omega\setminus C)$ has probability $0$, and $P(E)=1$ by claim 3 there. For $\omega\in E$ the sequence $(\ell_{h,n}(Y(\omega)))_{n}=(T_{n}(\omega))_{n}$ converges, since $\omega\in C$, to $T(\omega)$; hence $Y(\omega)\in D_{h}$ and $\ell_{h}(Y(\omega))=T(\omega)$ by The Paley-Wiener Functional of a Vector Relative to a Variance Sequence §functional. Thus

$$\ell_{h,n}\circ Y=T_{n}\ \text{on }G\ (n\in\mathbb{N}),\qquad\ell_{h}\circ Y=T\ \text{on }E,\qquad E\subseteq Y^{-1}(D_{h}),$$

where $\Omega\setminus G$ and $\Omega\setminus E$ are events of probability $0$; in particular each of these equalities holds $P$-almost everywhere, the exceptional set being contained in an event of probability $0$ and hence $P$-null, as A Property Holding Almost Everywhere requires.

Step 8 (almost everywhere convergence). Keep $h\in H_{c}$. By Step 2, $D_{h}\in\mathcal{B}(X)$, so $Y^{-1}(D_{h})\in\mathcal{F}$ and $\gamma_{c}(D_{h})=P(Y^{-1}(D_{h}))$ by Step 5. Since $E\subseteq Y^{-1}(D_{h})\subseteq\Omega$, monotonicity (claim 2 of Basic Properties of a Measure) gives $1=P(E)\le P(Y^{-1}(D_{h}))\le P(\Omega)=1$. Hence $\gamma_{c}(D_{h})=1$, which is the first assertion of claim 3.

Step 9 (integrability). Keep $h\in H_{c}$ and let $n\in\mathbb{N}$. The random variables $\ell_{h}\circ Y$ and $\ell_{h,n}\circ Y$ (Step 5, using Steps 1 and 3) agree with the square-integrable random variables $T$ and $T_{n}$ off the events $\Omega\setminus E$ and $\Omega\setminus G$ of probability $0$ (Step 7). By Almost Sure Equality Preserves Square-Integrability and the Mean-Square Norm, $\ell_{h}\circ Y$ and $\ell_{h,n}\circ Y$ are square-integrable, with

$$\mathbb{E}\bigl[(\ell_{h}\circ Y)^{2}\bigr]=\mathbb{E}[T^{2}].$$

Let $g$ be $\ell_{h}$ or $\ell_{h,n}$. Then $g\circ Y$ is integrable with respect to $P$, since square-integrable random variables are integrable by Square-Integrable Random Variables and the Mean-Square Inner Product; and $(g\circ Y)^{2}=g^{2}\circ Y$ is a nonnegative random variable with finite integral, hence integrable by the criterion of Measure Spaces and the Lebesgue Integral: Standing Notation §integral. The functions $g$ and $g^{2}$ are Borel by Steps 1 and 3 and claim 3 of Arithmetic, Absolute Values, and Pointwise Limits of Measurable Real-Valued Functions, so by Step 5 both $g$ and $g^{2}$ are integrable with respect to $\gamma_{c}$, and

$$\int_{X}g\,d\gamma_{c}=\int_{\Omega}g\circ Y\,dP,\qquad\int_{X}g^{2}\,d\gamma_{c}=\int_{\Omega}(g\circ Y)^{2}\,dP.$$

This is the second assertion of claim 3.

Step 10 (mean-square convergence). Keep $h\in H_{c}$ and $n\in\mathbb{N}$, and let $v=(\ell_{h}-\ell_{h,n})^{2}$, a nonnegative Borel function by Steps 1 and 3 and claims 2 and 3 of Arithmetic, Absolute Values, and Pointwise Limits of Measurable Real-Valued Functions. By Step 7, $v\circ Y=(T-T_{n})^{2}$ on $E$, so $P$-almost everywhere; both are nonnegative random variables (Step 5 and Square-Integrable Random Variables and the Mean-Square Inner Product). By Step 5 (nonnegative case), the almost-everywhere comparison The Lebesgue Integral and Null Sets: Almost-Everywhere Comparison, Markov's Inequality, and Dominated Convergence Almost Everywhere §comparison for nonnegative functions, and Step 6,

$$\int_{X}v\,d\gamma_{c}=\int_{\Omega}v\circ Y\,dP=\int_{\Omega}(T-T_{n})^{2}\,dP=\mathbb{E}\bigl[(T-T_{n})^{2}\bigr]=\sum_{k=n+1}^{\infty}\frac{h_{k}^{2}}{c_{k}},$$

the integrals of nonnegative functions being those of Measure Spaces and the Lebesgue Integral: Standing Notation §integral and the expectation that of Expectation, Variance, and Moments. This value is a real number, so $v$ is integrable with respect to $\gamma_{c}$ by the criterion of Measure Spaces and the Lebesgue Integral: Standing Notation §integral, and its integral in the sense of Integrable Function and the Lebesgue Integral is the same number, since $v^{+}=v$ and $v^{-}=0$, whose integral is $0$ by The Lebesgue Integral and Null Sets: Almost-Everywhere Comparison, Markov's Inequality, and Dominated Convergence Almost Everywhere §null-integral with the null set $\emptyset$. By Step 6 this value tends to $0$ as $n\to\infty$. This is the third assertion of claim 3, and with Steps 8 and 9 it completes claim 3.

Step 11 (mean and variance; claim 4). Keep $h\in H_{c}$. By Step 9 with $g=\ell_{h}$, then by The Lebesgue Integral and Null Sets: Almost-Everywhere Comparison, Markov's Inequality, and Dominated Convergence Almost Everywhere §comparison applied to the integrable random variable $T$ (integrable as a square-integrable random variable, by Square-Integrable Random Variables and the Mean-Square Inner Product) and to $\ell_{h}\circ Y$, which equals $T$ almost everywhere by Step 7, and finally by Step 6,

$$\int_{X}\ell_{h}\,d\gamma_{c}=\int_{\Omega}\ell_{h}\circ Y\,dP=\int_{\Omega}T\,dP=\mathbb{E}[T]=0.$$

By Step 9 with $g=\ell_{h}$ and the identity displayed there, and by Step 6,

$$\int_{X}(\ell_{h})^{2}\,d\gamma_{c}=\int_{\Omega}(\ell_{h}\circ Y)^{2}\,dP=\mathbb{E}\bigl[(\ell_{h}\circ Y)^{2}\bigr]=\mathbb{E}[T^{2}]=|h|_{c}^{2},$$

where the integral of the nonnegative integrable function $(\ell_{h}\circ Y)^{2}$ is its expectation by Expectation, Variance, and Moments (the two readings of the expectation agree, since the negative part is $0$, with integral $0$ by The Lebesgue Integral and Null Sets: Almost-Everywhere Comparison, Markov's Inequality, and Dominated Convergence Almost Everywhere §null-integral). This proves claim 4.