Proof of Expectations and Variances of Independent Random Variables

Throughout, the projections of $\mathbb{R}\times\mathbb{R}$ onto its factors are $\mathcal{B}(\mathbb{R})\otimes\mathcal{B}(\mathbb{R})$-measurable (The Product σ-Algebra: Rectangles, Generators, Sections, Projections, Pairings and the Swap (vi)), so the functions $h(x,y)=xy$ and $|h|(x,y)=|xy|=|x|\,|y|$ on $\mathbb{R}\times\mathbb{R}$ are measurable (Arithmetic of Measurable Real- and [0,∞]-Valued Maps (i)), and $|h|$, having nonnegative values, lies in $\mathcal{M}^{+}(\mathbb{R}\times\mathbb{R},\mathcal{B}(\mathbb{R})\otimes\mathcal{B}(\mathbb{R}))$ (Linearity and Monotonicity of the Lebesgue Integral, and the Integrability Criterion (vi)); likewise the identity $v(t)=t$ of $\mathbb{R}$ (Measurable Maps: Generators, Composition, Continuous Maps, Restrictions, and Criteria for Real- and [0,∞]-Valued Maps (iii)) and $u(t)=|t|$ are measurable, with $u\in\mathcal{M}^{+}(\mathbb{R},\mathcal{B}(\mathbb{R}))$. Laws are taken on $(\mathbb{R},\mathcal{B}(\mathbb{R}))$, and Independent Random Elements: Product Laws and Iterated Expectations is applied with $S=T=\mathbb{R}$.

Step 1: (i) for $X,Y\ge0$. As $X,Y\ge0$, we have $XY=|h|(X,Y)$, $X=u(X)$ and $Y=u(Y)$, so $\mathbb{E}X=\int u\,dP_{X}$ and $\mathbb{E}Y=\int u\,dP_{Y}$ (Expectations Computed from the Law (i)). For $x\in\mathbb{R}$ the section $|h|_{x}$ is $y\mapsto|x|\,u(y)$, so $\int|h|_{x}\,dP_{Y}=|x|\,\mathbb{E}Y$ (Basic Properties of the Integral of Nonnegative Measurable Maps (iv)). By Independent Random Elements: Product Laws and Iterated Expectations (ii),

$$\mathbb{E}[XY]=\int_{\mathbb{R}}\Bigl(\int_{\mathbb{R}}|h|_{x}\,dP_{Y}\Bigr)dP_{X}(x)=\int_{\mathbb{R}}c\,u\,dP_{X},\qquad c=\mathbb{E}Y\in[0,\infty].$$

If $c$ is real, the right side is $c\,\mathbb{E}X$ by Basic Properties of the Integral of Nonnegative Measurable Maps (iv). If $c=\infty$, then $\sup_{k}k=\infty$ in $[0,\infty]$ (The Integers in the Real Numbers: Greatest Elements, the Integer Part, and Archimedean Consequences (vii)), so the functions $k\,u$, $k\in\mathbb{N}$, increase pointwise to $\infty\cdot u$ (Suprema and Finite Sums in [0,∞] (ii)), and monotone convergence (Monotone Convergence Theorem (ii)) gives $\int\infty\cdot u\,dP_{X}=\sup_{k}k\,\mathbb{E}X=\infty\cdot\mathbb{E}X$. In both cases $\mathbb{E}[XY]=\mathbb{E}X\cdot\mathbb{E}Y$.

Step 2: (i) for integrable $X,Y$. The random variables $|X|=u(X)$ and $|Y|=u(Y)$ are independent by Independent Random Elements: Preimages, Functions and Blocks (iv), and $|XY|=|X|\,|Y|$, so Step 1 gives $\mathbb{E}|XY|=\mathbb{E}|X|\,\mathbb{E}|Y|<\infty$; hence the random variable $XY=h(X,Y)$ is integrable (Linearity and Monotonicity of the Lebesgue Integral, and the Integrability Criterion (iv)), and $h\in\mathcal{L}^{1}(P_{X}\otimes P_{Y})$ by Independent Random Elements: Product Laws and Iterated Expectations (iii). As $v(X)=X$ and $v(Y)=Y$, we have $v\in\mathcal{L}^{1}(P_{X})\cap\mathcal{L}^{1}(P_{Y})$ with $\int v\,dP_{X}=\mathbb{E}X$ and $\int v\,dP_{Y}=\mathbb{E}Y$ (Expectations Computed from the Law (ii)). For every $x\in\mathbb{R}$ the section $h_{x}=x\,v$ therefore lies in $\mathcal{L}^{1}(P_{Y})$ with integral $x\,\mathbb{E}Y$ (Linearity and Monotonicity of the Lebesgue Integral, and the Integrability Criterion (vii)), so the function $I$ of Independent Random Elements: Product Laws and Iterated Expectations (iii) is $I=(\mathbb{E}Y)\,v$, and, by the same lemma and linearity (Linearity and Monotonicity of the Lebesgue Integral, and the Integrability Criterion (vii)),

$$\mathbb{E}[XY]=\int_{\mathbb{R}}(\mathbb{E}Y)\,v\,dP_{X}=\mathbb{E}Y\int_{\mathbb{R}}v\,dP_{X}=\mathbb{E}X\,\mathbb{E}Y.$$

Proof of (ii). As $X,Y\in\mathcal{L}^{1}(P)$ (Lᵖ Inclusions and Constants on a Finite Measure Space (i)), the covariance formula (Rules for Expectations, Variances and Covariances (vii)) and (i) give

$$\operatorname{Cov}(X,Y)=\mathbb{E}[XY]-\mathbb{E}X\,\mathbb{E}Y=0.$$

Proof of (iii). For $r\in[n]$ let $T_{r}=\sum_{k=1}^{r}X_{k}$. We show by induction on $r$ (Induction for Predicative Formulas, and the Order of ℕ below 1 and below n+1 (i)) that for $r\in[n]$: $T_{r}\in\mathcal{L}^{2}(P)$, $\operatorname{Var}T_{r}=\sum_{k=1}^{r}\operatorname{Var}X_{k}$, and $\operatorname{Cov}(T_{r},X_{j})=0$ for every $j\in[n]$ with $j>r$. For $r=1$ we have $T_{1}=X_{1}$ (Iterated Operations: Recursion, Splitting, Reordering, Termwise Combination and Homomorphisms (i)), so the first two assertions are clear, and for $j>1$ the hypothesis gives $\operatorname{Cov}(T_{1},X_{j})=\operatorname{Cov}(X_{1},X_{j})=0$, as $j\neq1$. Let $r+1\in[n]$, so that $r\in[n]$, and assume the claim for $r$. Then $T_{r+1}=T_{r}+X_{r+1}$ (Iterated Operations: Recursion, Splitting, Reordering, Termwise Combination and Homomorphisms (i)), which lies in $\mathcal{L}^{2}(P)$ (Minkowski's Inequality (i)). By symmetry and bilinearity of the covariance (Rules for Expectations, Variances and Covariances (x)) and the claim for $r$ with $j=r+1$,

$$\operatorname{Var}T_{r+1}=\operatorname{Var}T_{r}+2\operatorname{Cov}(T_{r},X_{r+1})+\operatorname{Var}X_{r+1}=\sum_{k=1}^{r+1}\operatorname{Var}X_{k},$$

and for $j\in[n]$ with $j>r+1$, so that $j>r$ and $j\neq r+1$ (Induction for Predicative Formulas, and the Order of ℕ below 1 and below n+1 (iii)), the claim for $r$ and the hypothesis give

$$\operatorname{Cov}(T_{r+1},X_{j})=\operatorname{Cov}(T_{r},X_{j})+\operatorname{Cov}(X_{r+1},X_{j})=0.$$

The case $r=n$ is the assertion.

Proof of (iv). $X+Y\in\mathcal{L}^{2}(P;\mathbb{R}^{n})$ (Square-Integrable Vector-Valued Functions: Components, Dot Products and Almost-Everywhere Equality (iii)), its components are $X_{i}+Y_{i}$, and $X_{i},Y_{i}\in\mathcal{L}^{2}(P)$ for $i\in[n]$ (Square-Integrable Vector-Valued Functions: Components, Dot Products and Almost-Everywhere Equality (i)). Let $i,j\in[n]$. The coordinate functions $x\mapsto x_{i}$ and $x\mapsto x_{j}$ on $\mathbb{R}^{n}$ being measurable (Vector- and Matrix-Valued Measurable Maps: Components, Entries and Continuous Images (i)), Independent Random Elements: Preimages, Functions and Blocks (iv) shows that $X_{i}$ and $Y_{j}$ are independent, and also $X_{j}$ and $Y_{i}$; so $\operatorname{Cov}(X_{i},Y_{j})=0$ and $\operatorname{Cov}(Y_{i},X_{j})=\operatorname{Cov}(X_{j},Y_{i})=0$ by (ii) and the symmetry of the covariance (Rules for Expectations, Variances and Covariances (x)). By Variance and Covariance (iv) and Rules for Expectations, Variances and Covariances (x),

$$\operatorname{Cov}(X+Y)_{ij}=\operatorname{Cov}(X_{i},X_{j})+\operatorname{Cov}(X_{i},Y_{j})+\operatorname{Cov}(Y_{i},X_{j})+\operatorname{Cov}(Y_{i},Y_{j})=\operatorname{Cov}(X)_{ij}+\operatorname{Cov}(Y)_{ij}.$$