Proof of Factorisation of Random Variables Through a Measurable Map, the Variational Form of the Mean-Square Filtering Error, and Its Invariance Under the Joint Law

Throughout, measurability of a real-valued map is with respect to the named $\sigma$-algebra and $\mathcal{B}(\mathbb{R})$, and we use freely that constants, indicators of measurable sets, sums, scalar multiples, products, absolute values, maxima and pointwise limits of measurable real-valued maps are measurable (claims 1--5 of Arithmetic, Absolute Values, and Pointwise Limits of Measurable Real-Valued Functions).

Step 1: claim 1. Preimages commute with complements and countable unions: $\mathsf{D}^{-1}(\mathsf{Y})=\Omega$, $\Omega\setminus\mathsf{D}^{-1}(B)=\mathsf{D}^{-1}(\mathsf{Y}\setminus B)$, and $\bigcup_n\mathsf{D}^{-1}(B_n)=\mathsf{D}^{-1}(\bigcup_nB_n)$; since $\mathcal{Y}$ is a $\sigma$-algebra, $\sigma(\mathsf{D})$ is a $\sigma$-algebra on $\Omega$, and $\sigma(\mathsf{D})\subseteq\mathcal{F}$ because $\mathsf{D}$ is measurable with respect to $\mathcal{F}$ and $\mathcal{Y}$. That $\mathsf{D}$ is measurable with respect to $\sigma(\mathsf{D})$ and $\mathcal{Y}$ is the definition of $\sigma(\mathsf{D})$.

Let $\mathcal{H}$ be as in the statement, the family of sets $A\in\mathcal{F}$ for which there is $A'\in\sigma(\mathsf{D})$ with $P(A\triangle A')=0$. Then $\mathcal{H}$ is a $\sigma$-algebra: $\Omega\in\mathcal{H}$ (take $A'=\Omega$); if $A\in\mathcal{H}$ with witness $A'$ then $(\Omega\setminus A)\triangle(\Omega\setminus A')=A\triangle A'$, so $\Omega\setminus A\in\mathcal{H}$; and if $A_n\in\mathcal{H}$ with witnesses $A'_n$ then $\bigl(\bigcup_nA_n\bigr)\triangle\bigl(\bigcup_nA'_n\bigr)\subseteq\bigcup_n(A_n\triangle A'_n)$, a countable union of sets of $\mathcal{F}$ of probability $0$, hence of probability $0$ by countable subadditivity (Basic Properties of a Measure), so $\bigcup_nA_n\in\mathcal{H}$. Moreover $\sigma(\mathsf{D})\subseteq\mathcal{H}$ (take $A'=A$) and $\mathcal{N}\subseteq\mathcal{H}$ (take $A'=\emptyset$; then $A\triangle\emptyset=A$ has probability $0$). Hence the $\sigma$-algebra $\mathcal{G}_{\mathcal{N}}$ generated by $\sigma(\mathsf{D})\cup\mathcal{N}$ satisfies $\sigma(\mathsf{D})\subseteq\mathcal{G}_{\mathcal{N}}\subseteq\mathcal{H}\subseteq\mathcal{F}$.

Conversely $\mathcal{H}\subseteq\mathcal{G}_{\mathcal{N}}$: let $A\in\mathcal{H}$ with witness $A'\in\sigma(\mathsf{D})$. The sets $A\setminus A'$ and $A'\setminus A$ lie in $\mathcal{F}$ and are contained in $A\triangle A'$, hence have probability $0$ by monotonicity of $P$ (Basic Properties of a Measure), so both lie in $\mathcal{N}$. Since

$$A=\bigl(A'\setminus(A'\setminus A)\bigr)\cup(A\setminus A')$$

and $\mathcal{G}_{\mathcal{N}}$ is a $\sigma$-algebra containing $A'$, $A'\setminus A$ and $A\setminus A'$, we get $A\in\mathcal{G}_{\mathcal{N}}$. So $\mathcal{G}_{\mathcal{N}}=\mathcal{H}$, and in particular $\mathcal{G}_{\mathcal{N}}$ is $\mathsf{D}$-generated up to null sets.

Finally, if $\mathcal{G}$ is any $\sigma$-algebra on $\Omega$ that is $\mathsf{D}$-generated up to null sets, then by definition every $A\in\mathcal{G}$ lies in $\mathcal{F}$ and admits a witness in $\sigma(\mathsf{D})$, that is $\mathcal{G}\subseteq\mathcal{H}=\mathcal{G}_{\mathcal{N}}$; and conversely a $\sigma$-algebra $\mathcal{G}$ with $\sigma(\mathsf{D})\subseteq\mathcal{G}\subseteq\mathcal{H}$ is $\mathsf{D}$-generated up to null sets, again by the definition of $\mathcal{H}$. This proves the remaining assertions of claim 1.

Step 2: claim 2, the easy direction. Let $g:\mathsf{Y}\to\mathbb{R}$ be $\mathcal{Y}$-measurable and $Z=g\circ\mathsf{D}$. For $A\in\mathcal{B}(\mathbb{R})$ we have $Z^{-1}(A)=\mathsf{D}^{-1}\bigl(g^{-1}(A)\bigr)$ with $g^{-1}(A)\in\mathcal{Y}$, so $Z^{-1}(A)\in\sigma(\mathsf{D})$ and $Z$ is $\sigma(\mathsf{D})$-measurable.

Step 3: claim 2, the factorisation. Let $Z:\Omega\to\mathbb{R}$ be $\sigma(\mathsf{D})$-measurable. For a natural number $n$ and an integer $k$ with $|k|\le n2^{n}$ put

$$E_{n,k}=\bigl\{\omega\in\Omega:\ k2^{-n}\le Z(\omega)<(k+1)2^{-n}\bigr\},$$

an element of $\sigma(\mathsf{D})$, since it is the preimage under $Z$ of a Borel set. For fixed $n$ these sets are pairwise disjoint. Choose $B_{n,k}\in\mathcal{Y}$ with $E_{n,k}=\mathsf{D}^{-1}(B_{n,k})$, and replace $B_{n,k}$ by $B_{n,k}\setminus\bigcup_{k'<k}B_{n,k'}$ (a finite union, so the result lies in $\mathcal{Y}$); this does not change the preimage, because

$$\mathsf{D}^{-1}\Bigl(B_{n,k}\setminus\bigcup_{k'<k}B_{n,k'}\Bigr)=E_{n,k}\setminus\bigcup_{k'<k}E_{n,k'}=E_{n,k}$$

by the disjointness of the $E_{n,k}$. So we may and do assume that for each fixed $n$ the sets $B_{n,k}$ are pairwise disjoint. Put

$$g_n=\sum_{k=-n2^{n}}^{n2^{n}}k2^{-n}\,\mathbf{1}_{B_{n,k}},$$

a finite sum of constants times indicators of members of $\mathcal{Y}$, hence $\mathcal{Y}$-measurable.

Fix $\omega\in\Omega$ and put $y=\mathsf{D}(\omega)$. If $|Z(\omega)|\le n$, then $\omega\in E_{n,k}$ for exactly one $k$ with $|k|\le n2^{n}$, namely the integer $k$ with $k2^{-n}\le Z(\omega)<(k+1)2^{-n}$; then $y\in B_{n,k}$, and $y$ lies in no other $B_{n,k'}$ by disjointness, so $g_n(y)=k2^{-n}$ and $|g_n(y)-Z(\omega)|\le2^{-n}$. Since $Z(\omega)$ is a real number, $|Z(\omega)|\le n$ for all large $n$, so the sequence $\bigl(g_n(\mathsf{D}(\omega))\bigr)_{n}$ converges to $Z(\omega)$.

Let $L$ be the set of $y\in\mathsf{Y}$ for which $(g_n(y))_{n}$ is a Cauchy sequence of real numbers. Then

$$L=\bigcap_{k\ge1}\ \bigcup_{M\ge1}\ \bigcap_{n\ge M}\ \bigcap_{m\ge M}\bigl\{y:|g_n(y)-g_m(y)|\le1/k\bigr\},$$

a countable intersection of countable unions of countable intersections of members of $\mathcal{Y}$, hence $L\in\mathcal{Y}$. Put $h_n=g_n\mathbf{1}_{L}$, $\mathcal{Y}$-measurable. At every $y\in L$ the sequence $(h_n(y))_n=(g_n(y))_n$ converges to a real number by Every Cauchy Sequence of Real Numbers Converges; at every $y\notin L$ it is constantly $0$. So $(h_n(y))_n$ converges to a real number for every $y$, and by claim 5 of Arithmetic, Absolute Values, and Pointwise Limits of Measurable Real-Valued Functions the map $g(y)=\lim_nh_n(y)$ is $\mathcal{Y}$-measurable. For every $\omega$, $\mathsf{D}(\omega)\in L$ by the previous paragraph (a convergent sequence is Cauchy), so $g(\mathsf{D}(\omega))=\lim_ng_n(\mathsf{D}(\omega))=Z(\omega)$. Thus $Z=g\circ\mathsf{D}$, proving claim 2.

Step 4: claim 3. Let $Y$ be a conditional expectation of $X$ given $\sigma(\mathsf{D})$, which exists by Existence and Uniqueness of Conditional Expectation for Square-Integrable Random Variables. Conditions (i) and (ii) of Conditional Expectation of a Square-Integrable Random Variable for $\mathcal{G}$ hold: $Y$ is $\sigma(\mathsf{D})$-measurable, hence $\mathcal{G}$-measurable because $\sigma(\mathsf{D})\subseteq\mathcal{G}$, and $Y$ is square-integrable. For (iii), let $A\in\mathcal{G}$ and choose $A'\in\sigma(\mathsf{D})$ with $P(A\triangle A')=0$. Then $\mathbf{1}_{A}$ and $\mathbf{1}_{A'}$ differ only on $A\triangle A'$, so $\mathbf{1}_{A}$ and $\mathbf{1}_{A'}$ are almost surely equal; hence $X\mathbf{1}_{A}$ and $X\mathbf{1}_{A'}$ are almost surely equal integrable random variables and have the same expectation, and likewise for $Y$. Therefore

$$\mathbb{E}[X\mathbf{1}_{A}]=\mathbb{E}[X\mathbf{1}_{A'}]=\mathbb{E}[Y\mathbf{1}_{A'}]=\mathbb{E}[Y\mathbf{1}_{A}],$$

the middle equality by property 3 of Existence and Uniqueness of Conditional Expectation for Square-Integrable Random Variables for $\sigma(\mathsf{D})$. So $Y$ is a conditional expectation of $X$ given $\mathcal{G}$. By the uniqueness assertion of Existence and Uniqueness of Conditional Expectation for Square-Integrable Random Variables applied to $\mathcal{G}$, any conditional expectation of $X$ given $\mathcal{G}$ is almost surely equal to $Y$, and by the same assertion applied to $\sigma(\mathsf{D})$, so is any conditional expectation of $X$ given $\sigma(\mathsf{D})$. Almost surely equal square-integrable random variables are at mean-square distance $0$ by Square-Integrable Random Variables and the Mean-Square Inner Product, so $X-\mathbb{E}[X\mid\mathcal{G}]$ and $X-\mathbb{E}[X\mid\sigma(\mathsf{D})]$ are almost surely equal and the displayed expectations agree.

Step 5: claim 4. The zero map is $\mathcal{Y}$-measurable with $0\circ\mathsf{D}=0$ square-integrable, so $\mathcal{M}\neq\emptyset$ and the displayed set of real numbers is nonempty; it is bounded below by $0$, so it has a greatest lower bound.

Let $Y$ be a conditional expectation of $X$ given $\sigma(\mathsf{D})$. By claim 2 there is a $\mathcal{Y}$-measurable $g_{\star}$ with $Y=g_{\star}\circ\mathsf{D}$; since $Y$ is square-integrable, $g_{\star}\in\mathcal{M}$. Conversely, for every $g\in\mathcal{M}$ the random variable $g\circ\mathsf{D}$ is $\sigma(\mathsf{D})$-measurable by Step 2 and square-integrable, so property 1 of Existence and Uniqueness of Conditional Expectation for Square-Integrable Random Variables gives $\lVert X-Y\rVert_{2}\le\lVert X-g\circ\mathsf{D}\rVert_{2}$, that is $\mathbb{E}[(X-Y)^{2}]\le\mathbb{E}[(X-g\circ\mathsf{D})^{2}]$. Since $Y=g_{\star}\circ\mathsf{D}$ with $g_{\star}\in\mathcal{M}$, the value $\mathbb{E}[(X-Y)^{2}]$ belongs to the set and is a lower bound for it, so it is the greatest lower bound and it is attained. By claim 3 it equals $\mathbb{E}[(X-\mathbb{E}[X\mid\mathcal{G}])^{2}]$, which proves claim 4.

Step 6: claim 5. The map $(X,\mathsf{D}):\Omega\to\mathbb{R}\times\mathsf{Y}$ is measurable with respect to $\mathcal{F}$ and $\mathcal{B}(\mathbb{R})\otimes\mathcal{Y}$. Indeed, the family $\mathcal{E}$ of sets $C\subseteq\mathbb{R}\times\mathsf{Y}$ with $(X,\mathsf{D})^{-1}(C)\in\mathcal{F}$ is a $\sigma$-algebra on $\mathbb{R}\times\mathsf{Y}$, because preimages commute with complements and countable unions and $(X,\mathsf{D})^{-1}(\mathbb{R}\times\mathsf{Y})=\Omega$; and $\mathcal{E}$ contains every set $A\times B$ with $A\in\mathcal{B}(\mathbb{R})$ and $B\in\mathcal{Y}$, since $(X,\mathsf{D})^{-1}(A\times B)=X^{-1}(A)\cap\mathsf{D}^{-1}(B)\in\mathcal{F}$. Such sets generate $\mathcal{B}(\mathbb{R})\otimes\mathcal{Y}$ by Product Sigma-Algebra, and a $\sigma$-algebra containing a family contains the $\sigma$-algebra it generates, so $\mathcal{B}(\mathbb{R})\otimes\mathcal{Y}\subseteq\mathcal{E}$, which is the asserted measurability. The same argument applies to $(X',\mathsf{D}')$. Write $\mathcal{L}$ for the common image measure on $\mathcal{B}(\mathbb{R})\otimes\mathcal{Y}$, a probability measure by claim 1 of that lemma.

For a $\mathcal{Y}$-measurable $g:\mathsf{Y}\to\mathbb{R}$ define $F_g:\mathbb{R}\times\mathsf{Y}\to[0,\infty)$ by $F_g(a,y)=(a-g(y))^{2}$. The coordinate maps $(a,y)\mapsto a$ and $(a,y)\mapsto g(y)$ are measurable with respect to $\mathcal{B}(\mathbb{R})\otimes\mathcal{Y}$, the first because the preimage of $A\in\mathcal{B}(\mathbb{R})$ is $A\times\mathsf{Y}$ and the second because the preimage of $A$ is $\mathbb{R}\times g^{-1}(A)$; hence $F_g$ is measurable. Moreover $F_g\circ(X,\mathsf{D})=(X-g\circ\mathsf{D})^{2}$ and $F_g\circ(X',\mathsf{D}')=(X'-g\circ\mathsf{D}')^{2}$ pointwise. By the change of variables of claim 2 of Image Measures, Measures with Densities, and Change of Variables, applied to the nonnegative measurable $F_g$,

$$\mathbb{E}\bigl[(X-g\circ\mathsf{D})^{2}\bigr]=\int_{\mathbb{R}\times\mathsf{Y}}F_g\,d\mathcal{L}=\mathbb{E}'\bigl[(X'-g\circ\mathsf{D}')^{2}\bigr]\qquad\text{in }[0,\infty],$$

both outer expectations being integrals of nonnegative measurable maps in the sense of Lebesgue Integral of a Nonnegative Measurable Function. Taking $g$ the zero map shows $\mathbb{E}[X^{2}]=\mathbb{E}'[X'^{2}]$, so $X'$ is square-integrable if and only if $X$ is; and for general $g$ the displayed identity shows that $g\circ\mathsf{D}$ is square-integrable if and only if $g\circ\mathsf{D}'$ is, because $(g\circ\mathsf{D})^{2}\le2(X-g\circ\mathsf{D})^{2}+2X^{2}$ and $(X-g\circ\mathsf{D})^{2}\le2X^{2}+2(g\circ\mathsf{D})^{2}$ pointwise, with the symmetric bounds on the primed space, and expectation is monotone and additive on nonnegative random variables by Linearity and Monotonicity of the Lebesgue Integral. Hence the set $\mathcal{M}$ of claim 4 formed on $(\Omega,\mathcal{F},P)$ with $\mathsf{D}$ coincides with the set formed on $(\Omega',\mathcal{F}',P')$ with $\mathsf{D}'$, and for every $g$ in it the two expectations agree. By claim 4 applied on each space,

$$\mathbb{E}\Bigl[\bigl(X-\mathbb{E}[X\mid\mathcal{G}]\bigr)^{2}\Bigr]=\inf_{g\in\mathcal{M}}\mathbb{E}\bigl[(X-g\circ\mathsf{D})^{2}\bigr]=\inf_{g\in\mathcal{M}}\mathbb{E}'\bigl[(X'-g\circ\mathsf{D}')^{2}\bigr]=\mathbb{E}'\Bigl[\bigl(X'-\mathbb{E}'[X'\mid\mathcal{G}']\bigr)^{2}\Bigr],$$

the two greatest lower bounds being those of the same set of real numbers. $\blacksquare$