Conditional Expectation Minimizes Weighted Mean-Square Estimation Error

Let $(\Omega,\mathcal{F},P)$ be a probability space, let $\mathcal{G}$ be a sub-$\sigma$-algebra of $\mathcal{F}$, let $k\ge1$ be a natural number, and let $X=(X^1,\dots,X^k)$ be a tuple of square-integrable random variables on $(\Omega,\mathcal{F},P)$. For each $\gamma\in\{1,\dots,k\}$ fix a conditional expectation $M^\gamma$ of $X^\gamma$ given $\mathcal{G}$, which exists by the existence and uniqueness theorem for conditional expectation; write $M=(M^1,\dots,M^k)$ and $\varepsilon=(\varepsilon^1,\dots,\varepsilon^k)$ with $\varepsilon^\gamma=X^\gamma-M^\gamma$. Each $M^\gamma$ is $\mathcal{G}$-measurable and square-integrable by conditions (i)--(ii) of the conditional-expectation definition, and each $\varepsilon^\gamma$ is square-integrable by the closure properties of the square-integrability definition.

Let $R$ be a symmetric positive semidefinite real matrix with $k$ rows and $k$ columns and entries $R_{\gamma\delta}$. For tuples $U=(U^1,\dots,U^k)$ and $V=(V^1,\dots,V^k)$ of square-integrable random variables write $U\cdot(RV)$ for the dot product of $U$ with the matrix-vector product $RV$, applied componentwise at each point of $\Omega$; its index formula is

a finite linear combination of the products $U^\gamma V^\delta$, each integrable by the closure properties of the square-integrability definition; hence $U\cdot(RV)$ is integrable with expectation $\mathbb{E}[U\cdot(RV)]=\sum_{\gamma,\delta}R_{\gamma\delta}\,\mathbb{E}[U^\gamma V^\delta]$, by the linearity of the integral applied finitely many times; for $V=U$ this recovers claim 1 of the expected quadratic form lemma.

Then for every tuple $Y=(Y^1,\dots,Y^k)$ of square-integrable random variables on $(\Omega,\mathcal{F},P)$ each of which is almost surely equal to a $\mathcal{G}$-measurable square-integrable random variable, with $\mathcal{G}$-measurability as in the existence and uniqueness theorem, and with the tuples $Y-X$ and $Y-M$ formed componentwise (their components square-integrable by the closure properties):

1. (Orthogonal decomposition.)

2. (Optimality and attainment.) Consequently

The tuple $M$ is itself admissible in place of $Y$ --- each $M^\gamma$ being $\mathcal{G}$-measurable and square-integrable --- and the bound is attained by every admissible $Y$ whose components are almost surely equal to the respective $M^\gamma$; in particular the infimum of $\mathbb{E}[(Y-X)\cdot(R(Y-X))]$ over all admissible $Y$ is attained and equals $\mathbb{E}[\varepsilon\cdot(R\varepsilon)]$.