Conditional Expectation Minimizes Weighted Mean-Square Estimation Error

Let $(\Omega,\mathcal{F},P)$ be a \reftext{def:probability-space-random-variable-2026a}{probability space}, let $\mathcal{G}$ be a \reftext{def:independence-sigma-algebras-2026a}{sub-$\sigma$-algebra} of $\mathcal{F}$, let $k\ge1$ be a \reftext{def:natural-numbers-2026a}{natural number}, and let $X=(X^1,\dots,X^k)$ be a tuple of \reftext{def:square-integrable-mean-square-2026a}{square-integrable} random variables on $(\Omega,\mathcal{F},P)$. For each $\gamma\in\{1,\dots,k\}$ fix a \reftext{def:conditional-expectation-l2-2026a}{conditional expectation} $M^\gamma$ of $X^\gamma$ given $\mathcal{G}$, which exists by \reftext{thm:conditional-expectation-l2-2026a}{the existence and uniqueness theorem for conditional expectation}; write $M=(M^1,\dots,M^k)$ and $\varepsilon=(\varepsilon^1,\dots,\varepsilon^k)$ with $\varepsilon^\gamma=X^\gamma-M^\gamma$. Each $M^\gamma$ is $\mathcal{G}$-measurable and square-integrable by conditions (i)--(ii) of the conditional-expectation definition, and each $\varepsilon^\gamma$ is square-integrable by the closure properties of the \reftext{def:square-integrable-mean-square-2026a}{square-integrability definition}.

Let $R$ be a symmetric \reftext{def:positive-semidefinite-matrix-2026a}{positive semidefinite} real matrix with $k$ rows and $k$ columns and entries $R_{\gamma\delta}$. For tuples $U=(U^1,\dots,U^k)$ and $V=(V^1,\dots,V^k)$ of square-integrable random variables write $U\cdot(RV)$ for the \reftext{def:dot-product-orthogonality-rn-2026a}{dot product} of $U$ with the \reftext{def:matrix-vector-product-2026a}{matrix-vector product} $RV$, applied componentwise at each point of $\Omega$; its index formula is

a finite linear combination of the products $U^\gamma V^\delta$, each \reftext{def:lebesgue-integral-integrable-2026a}{integrable} by the closure properties of the square-integrability definition; hence $U\cdot(RV)$ is integrable with \reftext{def:expectation-variance-2026a}{expectation} $\mathbb{E}[U\cdot(RV)]=\sum_{\gamma,\delta}R_{\gamma\delta}\,\mathbb{E}[U^\gamma V^\delta]$, by \reftext{thm:linearity-monotonicity-integral-2026a}{the linearity of the integral} applied finitely many times; for $V=U$ this recovers claim 1 of \reftext{lem:expected-quadratic-form-2026a}{the expected quadratic form lemma}.

Then for every tuple $Y=(Y^1,\dots,Y^k)$ of square-integrable random variables on $(\Omega,\mathcal{F},P)$ each of which is \reftext{def:almost-surely-2026a}{almost surely} equal to a $\mathcal{G}$-measurable square-integrable random variable, with $\mathcal{G}$-measurability as in \reftext{thm:conditional-expectation-l2-2026a}{the existence and uniqueness theorem}, and with the tuples $Y-X$ and $Y-M$ formed componentwise (their components square-integrable by the closure properties):

\textbf{1. (Orthogonal decomposition.)}

\textbf{2. (Optimality and attainment.)} Consequently

The tuple $M$ is itself admissible in place of $Y$ --- each $M^\gamma$ being $\mathcal{G}$-measurable and square-integrable --- and the bound is attained by every admissible $Y$ whose components are almost surely equal to the respective $M^\gamma$; in particular the infimum of $\mathbb{E}[(Y-X)\cdot(R(Y-X))]$ over all admissible $Y$ is attained and equals $\mathbb{E}[\varepsilon\cdot(R\varepsilon)]$.