Proof of Conditional Expectation as the Best Mean-Square Predictor

Throughout, the field and order laws of $\mathbb{R}$ are used freely (The Structure of ℝ as a Field and an Ordered Ring (i), The Structure of ℝ as a Field and an Ordered Ring (ii), Rules of Arithmetic and Order in an Ordered Field (iv), Rules of Arithmetic and Order in an Ordered Field (vi)), and $\mathcal{G}$-measurability of a function into $\mathbb{R}$ or $\mathbb{R}^{n}$ is $\mathcal{G}/\mathcal{B}(\mathbb{R})$- or $\mathcal{G}/\mathcal{B}(\mathbb{R}^{n})$-measurability (Conditional Expectations: Notation (ii)). By Conditional Expectation (iii), $Y_{k}\in\mathbb{E}[X_{k}\mid\mathcal{G}]$ for $k\in[n]$. A function $\Omega\to\mathbb{R}^{n}$ lies in $\mathcal{L}^{2}(P;\mathbb{R}^{n})$ if and only if it is measurable with components in $\mathcal{L}^{2}(P)$ (Square-Integrable Vector-Valued Functions: Components, Dot Products and Almost-Everywhere Equality (i)); it is $\mathcal{G}/\mathcal{B}(\mathbb{R}^{n})$-measurable if and only if its components are $\mathcal{G}/\mathcal{B}(\mathbb{R})$-measurable (Vector- and Matrix-Valued Measurable Maps: Components, Entries and Continuous Images (i) on $(\Omega,\mathcal{G})$), so $Y$ is; sums, differences and images under a constant matrix of $\mathcal{G}/\mathcal{B}(\mathbb{R}^{n})$-measurable functions are $\mathcal{G}/\mathcal{B}(\mathbb{R}^{n})$-measurable (Sums, Products, Transposes, Dot Products and Norms of Measurable Vector- and Matrix-Valued Maps (i), Sums, Products, Transposes, Dot Products and Norms of Measurable Vector- and Matrix-Valued Maps (ii), Measurable Maps: Generators, Composition, Continuous Maps, Restrictions, and Criteria for Real- and [0,∞]-Valued Maps (iii)), and such functions are $\mathcal{F}/\mathcal{B}(\mathbb{R}^{n})$-measurable, as $\mathcal{G}\subseteq\mathcal{F}$.

Proof of (i). For $k\in[n]$, $X_{k}\in\mathcal{L}^{2}(P)$, so $Y_{k}\in\mathcal{L}^{2}(P)$ by the contraction in $\mathcal{L}^{2}$ (Conditional Jensen Inequality, Lᵖ Contraction and Convergence Theorems for Conditional Expectations (iii) with $p=2$). Hence $Y\in\mathcal{L}^{2}(P;\mathbb{R}^{n})$.

Proof of (ii). Let $k\in[n]$. The component $W_{k}\in\mathcal{L}^{2}(P)$ is $\mathcal{G}/\mathcal{B}(\mathbb{R})$-measurable, and $W_{k}X_{k},W_{k}Y_{k}\in\mathcal{L}^{1}(P)$ by the Cauchy-Schwarz inequality (Hölder's Inequality (ii)). Taking out what is known (Rules for Conditional Expectations (vii)) gives $W_{k}Y_{k}\in\mathbb{E}[W_{k}X_{k}\mid\mathcal{G}]$, so $\mathbb{E}[W_{k}Y_{k}]=\mathbb{E}[W_{k}X_{k}]$ (Rules for Conditional Expectations (ii)). As $X-Y\in\mathcal{L}^{2}(P;\mathbb{R}^{n})$ (Square-Integrable Vector-Valued Functions: Components, Dot Products and Almost-Everywhere Equality (iii)), Square-Integrable Vector-Valued Functions: Components, Dot Products and Almost-Everywhere Equality (ii) and Rules for Expectations, Variances and Covariances (iii) give that $(X-Y)\cdot W$ is integrable and

$$\mathbb{E}\bigl[(X-Y)\cdot W\bigr]=\sum_{k=1}^{n}\bigl(\mathbb{E}[X_{k}W_{k}]-\mathbb{E}[Y_{k}W_{k}]\bigr)=0 .$$

Proof of (iii). Matrix images. For $F\in\mathcal{L}^{2}(P;\mathbb{R}^{n})$ the function $RF$ is measurable, and $\lVert RF(\omega)\rVert^{2}\le c\,\lVert F(\omega)\rVert^{2}$ for every $\omega$, with $c=\sum_{i,j=1}^{n}R_{ij}^{2}$ (Algebraic Rules for Matrix Products, Matrix-Vector Products and Transposes (ix)); so $\int|RF|^{2}\,dP\le c\int|F|^{2}\,dP<\infty$ (Square-Integrable Vector-Valued Functions (i), Basic Properties of the Integral of Nonnegative Measurable Maps (iii), Basic Properties of the Integral of Nonnegative Measurable Maps (iv)), that is, $RF\in\mathcal{L}^{2}(P;\mathbb{R}^{n})$.

Expanding the error. Let $D=X-Y$ and $H=Y-W$, which lie in $\mathcal{L}^{2}(P;\mathbb{R}^{n})$ by (i) and Square-Integrable Vector-Valued Functions: Components, Dot Products and Almost-Everywhere Equality (iii); $H$ is $\mathcal{G}/\mathcal{B}(\mathbb{R}^{n})$-measurable, and $X-W=D+H$. For $u,v\in\mathbb{R}^{n}$, linearity of the matrix-vector product and bilinearity of the dot product (Algebraic Rules for Matrix Products, Matrix-Vector Products and Transposes (v), The Dot Product on $\mathbb{R}^n$ Is Symmetric, Bilinear and Positive Definite (ii), The Dot Product on $\mathbb{R}^n$ Is Symmetric, Bilinear and Positive Definite (iii)) give

$$(u+v)\cdot\bigl(R(u+v)\bigr)=u\cdot(Ru)+u\cdot(Rv)+v\cdot(Ru)+v\cdot(Rv),$$

and $v\cdot(Ru)=(Ru)\cdot v=u\cdot(R^{\top}v)=u\cdot(Rv)$ (The Dot Product on $\mathbb{R}^n$ Is Symmetric, Bilinear and Positive Definite (i), Algebraic Rules for Matrix Products, Matrix-Vector Products and Transposes (viii), and $R^{\top}=R$ by Symmetric Real Matrices and the Set $\mathcal{S}(n)$ (i)). Hence, pointwise on $\Omega$,

$$(X-W)\cdot\bigl(R(X-W)\bigr)=D\cdot(RD)+2\,D\cdot(RH)+H\cdot(RH).$$

By the matrix images and Square-Integrable Vector-Valued Functions: Components, Dot Products and Almost-Everywhere Equality (ii), the three functions on the right are integrable, so the left side is integrable and

$$\mathbb{E}\bigl[(X-W)\cdot\bigl(R(X-W)\bigr)\bigr]=\mathbb{E}\bigl[D\cdot(RD)\bigr]+2\,\mathbb{E}\bigl[D\cdot(RH)\bigr]+\mathbb{E}\bigl[H\cdot(RH)\bigr]$$

(Rules for Expectations, Variances and Covariances (iii)). Finally $RH\in\mathcal{L}^{2}(P;\mathbb{R}^{n})$ is $\mathcal{G}/\mathcal{B}(\mathbb{R}^{n})$-measurable, so (ii), applied with $RH$ in place of $W$, gives $\mathbb{E}[D\cdot(RH)]=\mathbb{E}[(X-Y)\cdot(RH)]=0$.

By (iii), in both remaining clauses equality holds if and only if $\mathbb{E}[H\cdot(RH)]=0$, with $H=Y-W$ as above.

Proof of (iv). As $R$ is positive semidefinite, $H(\omega)\cdot\bigl(RH(\omega)\bigr)\ge0$ for every $\omega$ (Positive Semidefinite and Positive Definite Real Matrices (i)), so $\mathbb{E}[H\cdot(RH)]\ge0$ (Rules for Expectations, Variances and Covariances (iv), Rules for Expectations, Variances and Covariances (ii)), and the inequality follows from (iii). If $W=Y$ a.s., then $H\cdot(RH)=0$ a.s., as $0\cdot(R0)=0$ (Algebraic Rules for Matrix Products, Matrix-Vector Products and Transposes (v), The Dot Product on $\mathbb{R}^n$ (i)), so $\mathbb{E}[H\cdot(RH)]=0$ (The Lebesgue Integral and Null Sets (iii)) and equality holds.

Proof of (v). By assumption $\mathbb{E}[H\cdot(RH)]=0$. As $R$ is positive definite, $H(\omega)\cdot\bigl(RH(\omega)\bigr)>0$ whenever $H(\omega)\neq0$ (Positive Semidefinite and Positive Definite Real Matrices (ii)), and $H(\omega)\cdot\bigl(RH(\omega)\bigr)=0$ whenever $H(\omega)=0$, as above. The function $H\cdot(RH)$ is therefore nonnegative, and it is measurable (Sums, Products, Transposes, Dot Products and Norms of Measurable Vector- and Matrix-Valued Maps (i)), so its integral as an element of $\mathcal{M}^{+}$ is $0$ (Linearity and Monotonicity of the Lebesgue Integral, and the Integrability Criterion (vi)) and $P\bigl(H\cdot(RH)>0\bigr)=0$ (The Lebesgue Integral and Null Sets (iv)). If $W(\omega)\neq Y(\omega)$, then $H(\omega)\neq0$ and so $H(\omega)\cdot\bigl(RH(\omega)\bigr)>0$. Thus $\{W\neq Y\}$ is contained in an event of probability $0$, that is, $W=Y$ a.s. (Null Set of a Measure (i), A Property Holding Almost Everywhere (i)).