Proof of Conditional Expectation Given Countably Many Jointly Gaussian Observations

Throughout, $X$ is square-integrable: $(X)$ is a Gaussian random vector by the definition of a jointly Gaussian family, so Claim 1 of Square-Integrability, Moments, and Covariance Matrix of a Gaussian Random Vector applies.

Part 1. By Sigma-Algebra Generated by Random Variables and Independence of Sigma-Algebras, $\mathcal{G}_r$ is the $\sigma$-algebra generated by the events $U_k^{-1}(B)$ with $k\le r$ and $B$ a Borel set. These generators lie in $\mathcal{G}_{r+1}$ and in $\mathcal{G}$, which are $\sigma$-algebras, so minimality of the generated $\sigma$-algebra gives $\mathcal{G}_r\subseteq\mathcal{G}_{r+1}$ and $\mathcal{G}_r\subseteq\mathcal{G}$; consequently $\sigma\bigl(\bigcup_r\mathcal{G}_r\bigr)\subseteq\mathcal{G}$. Conversely, every generator $U_k^{-1}(B)$ of $\mathcal{G}$ lies in $\mathcal{G}_k\subseteq\bigcup_r\mathcal{G}_r\subseteq\sigma\bigl(\bigcup_r\mathcal{G}_r\bigr)$, so minimality gives $\mathcal{G}\subseteq\sigma\bigl(\bigcup_r\mathcal{G}_r\bigr)$. Hence equality.

Part 2. For each $r$, the tuple $(X,U_1,\dots,U_r)$ is a Gaussian random vector by the definition of a jointly Gaussian family, so Conditional Expectation for Jointly Gaussian Random Variables is Affine provides real numbers $\beta_{r,0},\dots,\beta_{r,r}$ for which $Y_r=\beta_{r,0}+\sum_{k\le r}\beta_{r,k}U_k$ is a conditional expectation of $X$ given $\mathcal{G}_r$.

Part 3. By Part 1, the sequence $(\mathcal{G}_r)_{r\in\mathbb{N}}$ is a nondecreasing sequence of sub-$\sigma$-algebras of $\mathcal{F}$ with $\sigma\bigl(\bigcup_r\mathcal{G}_r\bigr)=\mathcal{G}$. Applying Levy's upward theorem in mean square to the square-integrable $X$, every choice of conditional expectations $Y_r$ of $X$ given $\mathcal{G}_r$ and every conditional expectation $Y$ of $X$ given $\mathcal{G}$ satisfy $\lVert Y_r-Y\rVert_{2}\to0$. The particular clause follows because, by Part 2 and the uniqueness assertion of Existence and Uniqueness of Conditional Expectation for Square-Integrable Random Variables, the $Y_r$ may be taken of the displayed affine form.

Part 4. Fix $k\in\mathbb{N}$ and a conditional expectation $Y$ of $X$ given $\mathcal{G}$, and for $r\ge k$ let $Y_r$ be the affine conditional expectation of Part 2. By part 3 of Conditional Expectation for Jointly Gaussian Random Variables is Affine, with the covariance,

$$\operatorname{Cov}(X-Y_r,\,U_k)=0\qquad(r\ge k).$$

Write $D_r=Y_r-Y$ and $C=U_k-\mathbb{E}[U_k]$ (square-integrable by Square-Integrability, Moments, and Covariance Matrix of a Gaussian Random Vector and the closure properties of Square-Integrable Random Variables and the Mean-Square Inner Product). Expanding the covariances with the linearity of expectation from Linearity and Monotonicity of the Lebesgue Integral,

$$\operatorname{Cov}(X-Y,\,U_k)-\operatorname{Cov}(X-Y_r,\,U_k)=\operatorname{Cov}(D_r,\,U_k)=\mathbb{E}\bigl[(D_r-\mathbb{E}[D_r])\,C\bigr].$$

By the Cauchy-Schwarz inequality, $|\mathbb{E}[D_r]|=|\mathbb{E}[D_r\cdot1]|\le\lVert D_r\rVert_{2}$ and hence, with the triangle inequality of the same lemma,

$$\bigl|\operatorname{Cov}(D_r,\,U_k)\bigr|\le\lVert D_r-\mathbb{E}[D_r]\rVert_{2}\,\lVert C\rVert_{2}\le2\,\lVert D_r\rVert_{2}\,\lVert C\rVert_{2}.$$

By Part 3, $\lVert D_r\rVert_{2}\to0$, so $\operatorname{Cov}(X-Y,\,U_k)=\lim_{r\to\infty}\operatorname{Cov}(X-Y_r,\,U_k)=0$. $\blacksquare$