Proof of Square-Integrable Random Vectors: Coordinates, Operations, Almost Sure Equality and the Mean-Square Form

Each result cited is universally quantified over the data in its own statement. Integrals over $\Omega$ are with respect to $P$; "the integral theorem" refers to Linearity and Monotonicity of the Lebesgue Integral, and the Integrability Criterion, whose claim 1 (additivity, homogeneity and monotonicity of the nonnegative integral) and claim 2 (linearity, $|\int f|\le\int|f|$ and monotonicity for integrable functions) are used throughout. Two preliminaries. First, a nonnegative random variable $V$ with $\mathbb{E}[V]<\infty$ is integrable, and its integral as an integrable function equals $\mathbb{E}[V]$: by Integrable Functions and Their Lebesgue Integral, $V^{+}=V$ and $V^{-}=0$, the nonnegative integral of the zero function being $0$ by claim 1 of the integral theorem with the factor $0$; conversely an integrable $V\ge0$ has $\mathbb{E}[V]=\int V^{+}\,dP<\infty$. Second, for a random vector $X$, $\lVert X\rVert^{2}=\sum_{i=1}^{d}X_{i}^{2}$ pointwise by claim 1 of Properties of the Euclidean Norm on $\mathbb{R}^n$: Square, Distance, Vanishing, Coordinate Bounds, Homogeneity, Triangle Inequalities and Expansion of a Square, and $X\cdot Y=\sum_{i=1}^{d}X_{i}Y_{i}$ pointwise by Difference, Dot Product, and Orthogonality in $\mathbb{R}^n$.

Claim 1. Suppose every $X_{i}$ is square-integrable, so that $\mathbb{E}[X_{i}^{2}]<\infty$ by Square-Integrable Random Variables and the Mean-Square Inner Product; then each $X_{i}^{2}$ is integrable by the first preliminary, so by Linearity of the Lebesgue Integral over a Finite Sum of Integrable Functions (i) and Linearity of the Lebesgue Integral over a Finite Sum of Integrable Functions (ii) (with all $c_{k}=1$) the sum $\lVert X\rVert^{2}$ is integrable with $\int\lVert X\rVert^{2}\,dP=\sum_{i=1}^{d}\mathbb{E}[X_{i}^{2}]$, and this integral is $\mathbb{E}[\lVert X\rVert^{2}]$, which is therefore finite. Conversely suppose $\mathbb{E}[\lVert X\rVert^{2}]<\infty$. For each $i$ and $\omega$, $|X_{i}(\omega)|\le\lVert X(\omega)\rVert$ by claim 4 of Properties of the Euclidean Norm on $\mathbb{R}^n$: Square, Distance, Vanishing, Coordinate Bounds, Homogeneity, Triangle Inequalities and Expansion of a Square, so $X_{i}(\omega)^{2}=|X_{i}(\omega)|^{2}\le\lVert X(\omega)\rVert^{2}$ by claim 1 of Nonnegativity of Squares in an Ordered Field and claim 2 of Monotonicity of Squaring on the Nonnegative Elements of an Ordered Field; monotonicity gives $\mathbb{E}[X_{i}^{2}]\le\mathbb{E}[\lVert X\rVert^{2}]<\infty$, so every $X_{i}$ is square-integrable, and the identity follows from the first part.

Claim 2. The maps $X+Y$, $aX$ and $X-Y$ are random vectors by Random Vectors: Coordinates, Functions, Pairs and Almost-Sure Equality (ii). By Pairs of Euclidean Points: Coordinate Projections, Pairings, the Product Measure on a Euclidean Space, Borel Norm Functions and Finite Sets (iv) applied to $x$ and $(-1)y$, and $\lVert(-1)y\rVert=\lVert y\rVert$ by claim 5 of Properties of the Euclidean Norm on $\mathbb{R}^n$: Square, Distance, Vanishing, Coordinate Bounds, Homogeneity, Triangle Inequalities and Expansion of a Square, one has $\lVert x+y\rVert^{2}=\lVert x-(-1)y\rVert^{2}\le2\lVert x\rVert^{2}+2\lVert y\rVert^{2}$ for all $x,y\in\mathbb{R}^{d}$, using $x-(-1)y=x+y$ from claims 3 and 2 of Euclidean Space $\mathbb{R}^n$ is a Real Vector Space and claim 5 of Elementary Identities in a Vector Space; pointwise on $\Omega$ this gives $\lVert X+Y\rVert^{2}\le2\lVert X\rVert^{2}+2\lVert Y\rVert^{2}$, so $\mathbb{E}[\lVert X+Y\rVert^{2}]\le2\mathbb{E}[\lVert X\rVert^{2}]+2\mathbb{E}[\lVert Y\rVert^{2}]<\infty$ by monotonicity, additivity and homogeneity of the nonnegative integral. Next $\lVert aX\rVert^{2}=a^{2}\lVert X\rVert^{2}$ pointwise by claim 5 of Properties of the Euclidean Norm on $\mathbb{R}^n$: Square, Distance, Vanishing, Coordinate Bounds, Homogeneity, Triangle Inequalities and Expansion of a Square and claim 1 of Nonnegativity of Squares in an Ordered Field, so $\mathbb{E}[\lVert aX\rVert^{2}]=a^{2}\mathbb{E}[\lVert X\rVert^{2}]<\infty$ by homogeneity; and $X-Y=X+(-1)Y$ by claims 2 and 3 of Euclidean Space $\mathbb{R}^n$ is a Real Vector Space, so the two previous cases give $\mathbb{E}[\lVert X-Y\rVert^{2}]<\infty$. For the dot product, $|X\cdot Y|\le\tfrac12(\lVert X\rVert^{2}+\lVert Y\rVert^{2})$ pointwise by Pairs of Euclidean Points: Coordinate Projections, Pairings, the Product Measure on a Euclidean Space, Borel Norm Functions and Finite Sets (iv), and the right side has finite nonnegative integral $\tfrac12(\mathbb{E}[\lVert X\rVert^{2}]+\mathbb{E}[\lVert Y\rVert^{2}])$ by claim 1 of the integral theorem, so $\int|X\cdot Y|\,dP<\infty$ by monotonicity and $X\cdot Y$ is integrable by the criterion of Integrable Functions and Their Lebesgue Integral. Each coordinate is square-integrable by claim 1, so each $X_{i}Y_{i}$ is integrable by Square-Integrable Random Variables and the Mean-Square Inner Product, and Linearity of the Lebesgue Integral over a Finite Sum of Integrable Functions (ii) applied to $X\cdot Y=\sum_{i=1}^{d}X_{i}Y_{i}$ gives $\mathbb{E}[X\cdot Y]=\sum_{i=1}^{d}\mathbb{E}[X_{i}Y_{i}]=\sum_{i=1}^{d}\langle X_{i},Y_{i}\rangle_{2}$. For the bound, the random variable $\lVert X\rVert$ of Random Vectors: Coordinates, Functions, Pairs and Almost-Sure Equality (ii) satisfies $\lVert X\rVert\cdot\lVert X\rVert=\lVert X\rVert^{2}$, so it is square-integrable with mean-square norm $\lVert\,\lVert X\rVert\,\rVert_{2}=\sqrt{\mathbb{E}[\lVert X\rVert^{2}]}$ by Square-Integrable Random Variables and the Mean-Square Inner Product, and likewise for $\lVert Y\rVert$. By claim 2 of the integral theorem, monotonicity of the nonnegative integral applied to $|X\cdot Y|\le\lVert X\rVert\,\lVert Y\rVert$ (Pairs of Euclidean Points: Coordinate Projections, Pairings, the Product Measure on a Euclidean Space, Borel Norm Functions and Finite Sets (iv)), and claim 1 of Cauchy-Schwarz and Triangle Inequalities for the Mean-Square Norm,

$$|\mathbb{E}[X\cdot Y]|\le\mathbb{E}\bigl[|X\cdot Y|\bigr]\le\mathbb{E}\bigl[\lVert X\rVert\,\lVert Y\rVert\bigr]\le\sqrt{\mathbb{E}[\lVert X\rVert^{2}]}\,\sqrt{\mathbb{E}[\lVert Y\rVert^{2}]}\ ,$$

where the middle expectation is the integral of the integrable product $\lVert X\rVert\,\lVert Y\rVert$ (integrable by Square-Integrable Random Variables and the Mean-Square Inner Product), which is nonnegative, so that the first preliminary identifies its two readings. Finally $X\cdot X=\lVert X\rVert^{2}$ pointwise by claim 1 of Properties of the Euclidean Norm on $\mathbb{R}^n$: Square, Distance, Vanishing, Coordinate Bounds, Homogeneity, Triangle Inequalities and Expansion of a Square, so $\mathbb{E}[X\cdot X]=\mathbb{E}[\lVert X\rVert^{2}]$ by the first preliminary.

Claim 3. Since $\{X=X\}=\Omega$ and $P(\Omega)=1$, $\sim_{P}$ is reflexive, and it is symmetric because $\{X=Y\}=\{Y=X\}$. For transitivity let $X\sim_{P}Y$ and $Y\sim_{P}Z$, and put $E=\{X=Y\}\cap\{Y=Z\}$, an event by Random Vectors: Coordinates, Functions, Pairs and Almost-Sure Equality (iv) and σ-Algebra and Measurable Space. By claim 3 of Basic Properties of a Measure, $P(\Omega\setminus\{X=Y\})=1-1=0$ and $P(\Omega\setminus\{Y=Z\})=0$; as $\Omega\setminus E$ is the union of these two sets, claim 4 there (applied to the sequence consisting of these two sets followed by empty sets) gives $P(\Omega\setminus E)\le0+0=0$, so $P(E)=1$ by claim 3 again. Since $E\subseteq\{X=Z\}$, claim 2 there gives $1=P(E)\le P(X=Z)\le1$, so $X\sim_{P}Z$. The three properties yield the assertions about the sets $[X]$ in the usual way: if $Z\in[X]\cap[Y]$ then $X\sim_{P}Z$ and $Z\sim_{P}Y$ by symmetry, so $X\sim_{P}Y$; if $X\sim_{P}Y$ then every $X'\sim_{P}X$ satisfies $X'\sim_{P}Y$ and conversely, so $[X]=[Y]$; and if $[X]=[Y]$ then $X\in[X]=[Y]$ by reflexivity, so $X\sim_{P}Y$. Now let $X\sim_{P}X'$ and $Y\sim_{P}Y'$, and let $E'=\{X=X'\}\cap\{Y=Y'\}$, of probability $1$ by the argument just given. On $E'$ one has $X+Y=X'+Y'$ and $aX=aX'$ pointwise, so $E'\subseteq\{X+Y=X'+Y'\}$ and $E'\subseteq\{aX=aX'\}$, both events by Random Vectors: Coordinates, Functions, Pairs and Almost-Sure Equality (iv), and claim 2 of Basic Properties of a Measure gives $X+Y\sim_{P}X'+Y'$ and $aX\sim_{P}aX'$. The nonnegative random variables $\lVert X\rVert^{2}$ and $\lVert X'\rVert^{2}$ agree on $\{X=X'\}$, whose complement is an event of probability $0$, so they agree $P$-almost everywhere and $\mathbb{E}[\lVert X\rVert^{2}]=\mathbb{E}[\lVert X'\rVert^{2}]$ by The Lebesgue Integral and Null Sets (iii). If moreover $\mathbb{E}[\lVert X\rVert^{2}]<\infty$ and $\mathbb{E}[\lVert Y\rVert^{2}]<\infty$, then $X\cdot Y$ is integrable by claim 2 and agrees with $X'\cdot Y'$ on $E'$, hence almost everywhere, so $\mathbb{E}[X\cdot Y]=\mathbb{E}[X'\cdot Y']$ by the same clause.

Claim 4. By claim 2, $U\cdot W$, $V\cdot W$, $(U+V)\cdot W$ and $(aU)\cdot W$ are integrable ($U+V$ and $aU$ having finite second moment by claim 2). Pointwise, $(U+V)\cdot W=U\cdot W+V\cdot W$ by claim 2 of The Dot Product on $\mathbb{R}^n$ Is Symmetric, Bilinear and Positive Definite, $(aU)\cdot W=a\,(U\cdot W)$ by claim 4 there, and $U\cdot V=V\cdot U$ by claim 1 there; the first two identities pass to expectations by claim 2 of the integral theorem, and the third is immediate. By claim 2, $\mathbb{E}[U\cdot U]=\mathbb{E}[\lVert U\rVert^{2}]$, which is nonnegative as a nonnegative integral. By The Lebesgue Integral and Null Sets (iv), $\mathbb{E}[\lVert U\rVert^{2}]=0$ if and only if $\lVert U\rVert^{2}=0$ almost everywhere. For each $\omega$, $\lVert U(\omega)\rVert^{2}=0$ holds exactly when $\lVert U(\omega)\rVert=0$ (claims 1 and 3 of Zero Products and Elementary Identities in a Field), exactly when $U(\omega)=0_{\mathbb{R}^{d}}$ (claim 3 of Properties of the Euclidean Norm on $\mathbb{R}^n$: Square, Distance, Vanishing, Coordinate Bounds, Homogeneity, Triangle Inequalities and Expansion of a Square), that is, exactly when $\omega\in\{U=\mathbf{0}\}$. The set $\{U=\mathbf{0}\}$ is an event by Random Vectors: Coordinates, Functions, Pairs and Almost-Sure Equality (iv); so $\lVert U\rVert^{2}=0$ almost everywhere exactly when $\Omega\setminus\{U=\mathbf{0}\}$ is contained in an event of probability $0$, which by claim 2 of Basic Properties of a Measure holds exactly when $P(\Omega\setminus\{U=\mathbf{0}\})=0$, that is, by claim 3 there, exactly when $P(U=\mathbf{0})=1$, which is $U\sim_{P}\mathbf{0}$.

Claim 5. By claim 3, if $X\sim_{P}X'$ and $Y\sim_{P}Y'$ with $X,Y\in\mathbf{L}^{2}$, then $X',Y'\in\mathbf{L}^{2}$ (as $\mathbb{E}[\lVert X'\rVert^{2}]=\mathbb{E}[\lVert X\rVert^{2}]$), $X+Y\sim_{P}X'+Y'$ and $aX\sim_{P}aX'$, and these sums and multiples lie in $\mathbf{L}^{2}$ by claim 2; so $[X+Y]=[X'+Y']$ and $[aX]=[aX']$ by claim 3, and the operations are well defined. Also $\mathbf{0}\in\mathbf{L}^{2}$, since $\lVert\mathbf{0}\rVert^{2}$ is the zero function by claim 3 of Properties of the Euclidean Norm on $\mathbb{R}^n$: Square, Distance, Vanishing, Coordinate Bounds, Homogeneity, Triangle Inequalities and Expansion of a Square, whose integral is $0$ as noted in the first preliminary, and $(-1)X\in\mathbf{L}^{2}$ by claim 2. The axioms of Vector Space over a Field for $L^{2}$ are of two kinds. Those asserting identities (associativity and commutativity of addition, and the four axioms for scalar multiplication) concern elements built from classes by the two operations; through $[X]+[Y]=[X+Y]$ and $a[X]=[aX]$ each side is the class of a random vector built from representatives by the pointwise operations, and two random vectors that agree pointwise have the same class by reflexivity, so each such axiom reduces to the corresponding pointwise identity in $\mathbb{R}^{d}$, which holds by Euclidean Space $\mathbb{R}^n$ is a Real Vector Space. The two existence axioms are witnessed as follows: $[\mathbf{0}]$ is a zero vector because $X+\mathbf{0}=X$ pointwise, the origin being the zero vector of $\mathbb{R}^{d}$ by Euclidean Space $\mathbb{R}^n$ is a Real Vector Space; and $[(-1)X]$ is an additive inverse of $[X]$ because $X+(-1)X=\mathbf{0}$ pointwise, by claim 5 of Elementary Identities in a Vector Space and the inverse axiom in $\mathbb{R}^{d}$. Hence $L^{2}$ is a real vector space with zero vector $[\mathbf{0}]$ and $-[X]=[(-1)X]$.