Proof of Conditional Expectation Minimizes Weighted Mean-Square Estimation Error

Reduction to exactly measurable estimators. For each $\gamma$ fix a $\mathcal{G}$-measurable square-integrable random variable $\tilde{Y}^\gamma$ with $Y^\gamma$ almost surely equal to $\tilde{Y}^\gamma$, as the statement provides, and write $\tilde{Y}=(\tilde{Y}^1,\dots,\tilde{Y}^k)$. For square-integrable random variables $U$, $\tilde{U}$, $V$ with $U$ almost surely equal to $\tilde{U}$: $U-\tilde{U}$ is square-integrable by the closure properties of the square-integrability definition, $\lVert U-\tilde{U}\rVert_{2}=0$ by the null-equivalence statement there, $\big|\mathbb{E}[(U-\tilde{U})V]\big|\le\lVert U-\tilde{U}\rVert_{2}\lVert V\rVert_{2}=0$ by the Cauchy--Schwarz inequality, and $\mathbb{E}[UV]-\mathbb{E}[\tilde{U}V]=\mathbb{E}[(U-\tilde{U})V]$ by the linearity of the integral; hence $\mathbb{E}[UV]=\mathbb{E}[\tilde{U}V]$. Every expectation in conclusions 1 and 2 is, by the index formula and linearity of the integral recorded in the statement, a finite linear combination of expectations of products of components; replacing $Y^\gamma$ by $\tilde{Y}^\gamma$ one factor at a time and applying the display above finitely many times leaves each such expectation unchanged. It therefore suffices to prove both conclusions with $Y$ replaced by $\tilde{Y}$. Write $D^\gamma=\tilde{Y}^\gamma-M^\gamma$ and $D=(D^1,\dots,D^k)$.

Measurability and integrability. Each $M^\gamma$ is $\mathcal{G}$-measurable and square-integrable by conditions (i)--(ii) of the definition of a conditional expectation, and each $\varepsilon^\gamma$ is square-integrable as recorded in the statement. Each $D^\gamma$ is square-integrable by the closure properties of the square-integrability definition, and is $\mathcal{G}$-measurable by conclusion 2 of the closed mean-square span lemma, applied with the family $\mathcal{C}=\{\tilde{Y}^\gamma,M^\gamma\}$ and the sub-$\sigma$-algebra $\mathcal{G}$: the first sentence of that conclusion gives that the finite linear combination $\tilde{Y}^\gamma-M^\gamma$ is $\mathcal{G}$-measurable. All products $D^\gamma D^\delta$, $D^\gamma\varepsilon^\delta$, and $\varepsilon^\gamma\varepsilon^\delta$ are integrable by the closure properties.

Pointwise decomposition. At every $\omega\in\Omega$, $\tilde{Y}^\gamma-X^\gamma=D^\gamma-\varepsilon^\gamma$ for each $\gamma$, so by the index formula of the statement

where the two cross sums are combined using the symmetry of $R$: relabelling the summation indices, $\sum_{\gamma,\delta}R_{\gamma\delta}\varepsilon^\gamma D^\delta=\sum_{\gamma,\delta}R_{\delta\gamma}\varepsilon^\delta D^\gamma=\sum_{\gamma,\delta}R_{\gamma\delta}D^\gamma\varepsilon^\delta$.

Vanishing of the cross term. Taking expectations and applying the linearity of the integral finitely many times over the terms of the sums,

and $\mathbb{E}[(\tilde{Y}-M)\cdot(R(\tilde{Y}-M))]=\mathbb{E}[D\cdot(RD)]$ by the definition of $D$. For all $\gamma,\delta$: the random variable $M^\delta$ is $\mathcal{G}$-measurable and square-integrable by conditions (i)--(ii) of the conditional-expectation definition and satisfies the averaging property (iii) there, which is property 3 of the existence and uniqueness theorem, hence also the orthogonality property 2, by the equivalence of properties 1--3 recorded in that theorem applied with $X^\delta$ in place of $X$; since $D^\gamma$ is a $\mathcal{G}$-measurable square-integrable random variable,

This proves conclusion 1.

Optimality and attainment. At every $\omega\in\Omega$, the value of the random variable $D\cdot(RD)$ is $\sum_{\gamma,\delta}R_{\gamma\delta}D^\gamma(\omega)D^\delta(\omega)=D(\omega)\cdot\big(R\,D(\omega)\big)$ by the index formulas of the dot product and the matrix-vector product, and $D(\omega)$ is a point of $\mathbb{R}^k$, so this value is $\ge0$ because $R$ is positive semidefinite. By the monotonicity of the integral in the linearity and monotonicity theorem, $\mathbb{E}[D\cdot(RD)]\ge0$, and the inequality of conclusion 2 follows from conclusion 1. The tuple $M$ is admissible in place of $Y$, each $M^\gamma$ being $\mathcal{G}$-measurable and square-integrable. If each $Y^\gamma$ is almost surely equal to $M^\gamma$, then each $D^\gamma=\tilde{Y}^\gamma-M^\gamma$ is almost surely equal to $0$ (the events $\{\tilde{Y}^\gamma\ne Y^\gamma\}$ and $\{Y^\gamma\ne M^\gamma\}$ lying in a common event of probability $0$ by countable additivity), so $\lVert D^\gamma\rVert_{2}=0$ by the null-equivalence statement of the square-integrability definition, and $\big|\mathbb{E}[D^\gamma D^\delta]\big|\le\lVert D^\gamma\rVert_{2}\lVert D^\delta\rVert_{2}=0$ for all $\gamma,\delta$ by the Cauchy--Schwarz inequality; hence, by the same finite application of linearity, $\mathbb{E}[D\cdot(RD)]=\sum_{\gamma,\delta}R_{\gamma\delta}\mathbb{E}[D^\gamma D^\delta]=0$, and by conclusion 1 the bound is attained. Together with the inequality this identifies the infimum over admissible $Y$ as $\mathbb{E}[\varepsilon\cdot(R\varepsilon)]$, attained at $M$. $\square$