Proof of Square-Integrable Random Vectors: Coordinates, Operations, Almost Sure Equality and the Mean-Square Form

Each result cited is universally quantified over the data in its own statement. Integrals over $\Omega$ are with respect to $P$; "the integral theorem" refers to Linearity and Monotonicity of the Lebesgue Integral, whose claim 1 (additivity, homogeneity and monotonicity of the nonnegative integral) and claim 2 (linearity, $|\int f|\le\int|f|$ and monotonicity for integrable functions) are used throughout. Two preliminaries. First, a nonnegative random variable $V$ with $\mathbb{E}[V]<\infty$ is integrable, and its integral as an integrable function equals $\mathbb{E}[V]$: by Integrable Function and the Lebesgue Integral, $V^{+}=V$ and $V^{-}=0$, the nonnegative integral of the zero function being $0$ by claim 1 of the integral theorem with the factor $0$; conversely an integrable $V\ge0$ has $\mathbb{E}[V]=\int V^{+}\,dP<\infty$. Second, for a random vector $X$, $\lVert X\rVert^{2}=\sum_{i=1}^{d}X_{i}^{2}$ pointwise by claim 1 of Elementary Properties of the Euclidean Norm on $\mathbb{R}^n$, and $X\cdot Y=\sum_{i=1}^{d}X_{i}Y_{i}$ pointwise by Difference, Dot Product, and Orthogonality in $\mathbb{R}^n$.

Claim 1. Suppose every $X_{i}$ is square-integrable, so that $\mathbb{E}[X_{i}^{2}]<\infty$ by Square-Integrable Random Variables and the Mean-Square Inner Product; then each $X_{i}^{2}$ is integrable by the first preliminary, so by Linearity of the Lebesgue Integral over a Finite Sum of Integrable Functions §integrable and Linearity of the Lebesgue Integral over a Finite Sum of Integrable Functions §linear (with all $c_{k}=1$) the sum $\lVert X\rVert^{2}$ is integrable with $\int\lVert X\rVert^{2}\,dP=\sum_{i=1}^{d}\mathbb{E}[X_{i}^{2}]$, and this integral is $\mathbb{E}[\lVert X\rVert^{2}]$, which is therefore finite. Conversely suppose $\mathbb{E}[\lVert X\rVert^{2}]<\infty$. For each $i$ and $\omega$, $|X_{i}(\omega)|\le\lVert X(\omega)\rVert$ by claim 4 of Elementary Properties of the Euclidean Norm on $\mathbb{R}^n$, so $X_{i}(\omega)^{2}=|X_{i}(\omega)|^{2}\le\lVert X(\omega)\rVert^{2}$ by claim 1 of Nonnegativity of Squares in an Ordered Field and claim 2 of Monotonicity of Squaring on the Nonnegative Elements of an Ordered Field; monotonicity gives $\mathbb{E}[X_{i}^{2}]\le\mathbb{E}[\lVert X\rVert^{2}]<\infty$, so every $X_{i}$ is square-integrable, and the identity follows from the first part.

Claim 2. The maps $X+Y$, $aX$ and $X-Y$ are random vectors by Basic Properties of Random Vectors: Coordinates, Borel Images and Arithmetic, Change of Variables, Almost Sure Equality and Pairs §composition. By Pairs of Euclidean Points: Coordinate Projections, Pairings, the Product Measure on a Euclidean Space, Borel Norm Functions and Finite Sets §functions applied to $x$ and $(-1)y$, and $\lVert(-1)y\rVert=\lVert y\rVert$ by claim 5 of Elementary Properties of the Euclidean Norm on $\mathbb{R}^n$, one has $\lVert x+y\rVert^{2}=\lVert x-(-1)y\rVert^{2}\le2\lVert x\rVert^{2}+2\lVert y\rVert^{2}$ for all $x,y\in\mathbb{R}^{d}$, using $x-(-1)y=x+y$ from claims 3 and 2 of Euclidean Space $\mathbb{R}^n$ is a Real Vector Space and claim 5 of Elementary Identities in a Vector Space; pointwise on $\Omega$ this gives $\lVert X+Y\rVert^{2}\le2\lVert X\rVert^{2}+2\lVert Y\rVert^{2}$, so $\mathbb{E}[\lVert X+Y\rVert^{2}]\le2\mathbb{E}[\lVert X\rVert^{2}]+2\mathbb{E}[\lVert Y\rVert^{2}]<\infty$ by monotonicity, additivity and homogeneity of the nonnegative integral. Next $\lVert aX\rVert^{2}=a^{2}\lVert X\rVert^{2}$ pointwise by claim 5 of Elementary Properties of the Euclidean Norm on $\mathbb{R}^n$ and claim 1 of Nonnegativity of Squares in an Ordered Field, so $\mathbb{E}[\lVert aX\rVert^{2}]=a^{2}\mathbb{E}[\lVert X\rVert^{2}]<\infty$ by homogeneity; and $X-Y=X+(-1)Y$ by claims 2 and 3 of Euclidean Space $\mathbb{R}^n$ is a Real Vector Space, so the two previous cases give $\mathbb{E}[\lVert X-Y\rVert^{2}]<\infty$. For the dot product, $|X\cdot Y|\le\tfrac12(\lVert X\rVert^{2}+\lVert Y\rVert^{2})$ pointwise by Pairs of Euclidean Points: Coordinate Projections, Pairings, the Product Measure on a Euclidean Space, Borel Norm Functions and Finite Sets §functions, and the right side has finite nonnegative integral $\tfrac12(\mathbb{E}[\lVert X\rVert^{2}]+\mathbb{E}[\lVert Y\rVert^{2}])$ by claim 1 of the integral theorem, so $\int|X\cdot Y|\,dP<\infty$ by monotonicity and $X\cdot Y$ is integrable by the criterion of Integrable Function and the Lebesgue Integral. Each coordinate is square-integrable by claim 1, so each $X_{i}Y_{i}$ is integrable by Square-Integrable Random Variables and the Mean-Square Inner Product, and Linearity of the Lebesgue Integral over a Finite Sum of Integrable Functions §linear applied to $X\cdot Y=\sum_{i=1}^{d}X_{i}Y_{i}$ gives $\mathbb{E}[X\cdot Y]=\sum_{i=1}^{d}\mathbb{E}[X_{i}Y_{i}]=\sum_{i=1}^{d}\langle X_{i},Y_{i}\rangle_{2}$. For the bound, the random variable $\lVert X\rVert$ of Basic Properties of Random Vectors: Coordinates, Borel Images and Arithmetic, Change of Variables, Almost Sure Equality and Pairs §composition satisfies $\lVert X\rVert\cdot\lVert X\rVert=\lVert X\rVert^{2}$, so it is square-integrable with mean-square norm $\lVert\,\lVert X\rVert\,\rVert_{2}=\sqrt{\mathbb{E}[\lVert X\rVert^{2}]}$ by Square-Integrable Random Variables and the Mean-Square Inner Product, and likewise for $\lVert Y\rVert$. By claim 2 of the integral theorem, monotonicity of the nonnegative integral applied to $|X\cdot Y|\le\lVert X\rVert\,\lVert Y\rVert$ (Pairs of Euclidean Points: Coordinate Projections, Pairings, the Product Measure on a Euclidean Space, Borel Norm Functions and Finite Sets §functions), and claim 1 of Cauchy-Schwarz and Triangle Inequalities for the Mean-Square Norm,

where the middle expectation is the integral of the integrable product $\lVert X\rVert\,\lVert Y\rVert$ (integrable by Square-Integrable Random Variables and the Mean-Square Inner Product), which is nonnegative, so that the first preliminary identifies its two readings. Finally $X\cdot X=\lVert X\rVert^{2}$ pointwise by claim 1 of Elementary Properties of the Euclidean Norm on $\mathbb{R}^n$, so $\mathbb{E}[X\cdot X]=\mathbb{E}[\lVert X\rVert^{2}]$ by the first preliminary.

Claim 3. Since $\{X=X\}=\Omega$ and $P(\Omega)=1$, $\sim_{P}$ is reflexive, and it is symmetric because $\{X=Y\}=\{Y=X\}$. For transitivity let $X\sim_{P}Y$ and $Y\sim_{P}Z$, and put $E=\{X=Y\}\cap\{Y=Z\}$, an event by Basic Properties of Random Vectors: Coordinates, Borel Images and Arithmetic, Change of Variables, Almost Sure Equality and Pairs §almost-sure and Sigma-Algebra and Measurable Space. By claim 3 of Basic Properties of a Measure, $P(\Omega\setminus\{X=Y\})=1-1=0$ and $P(\Omega\setminus\{Y=Z\})=0$; as $\Omega\setminus E$ is the union of these two sets, claim 4 there (applied to the sequence consisting of these two sets followed by empty sets) gives $P(\Omega\setminus E)\le0+0=0$, so $P(E)=1$ by claim 3 again. Since $E\subseteq\{X=Z\}$, claim 2 there gives $1=P(E)\le P(X=Z)\le1$, so $X\sim_{P}Z$. The three properties yield the assertions about the sets $[X]$ in the usual way: if $Z\in[X]\cap[Y]$ then $X\sim_{P}Z$ and $Z\sim_{P}Y$ by symmetry, so $X\sim_{P}Y$; if $X\sim_{P}Y$ then every $X'\sim_{P}X$ satisfies $X'\sim_{P}Y$ and conversely, so $[X]=[Y]$; and if $[X]=[Y]$ then $X\in[X]=[Y]$ by reflexivity, so $X\sim_{P}Y$. Now let $X\sim_{P}X'$ and $Y\sim_{P}Y'$, and let $E'=\{X=X'\}\cap\{Y=Y'\}$, of probability $1$ by the argument just given. On $E'$ one has $X+Y=X'+Y'$ and $aX=aX'$ pointwise, so $E'\subseteq\{X+Y=X'+Y'\}$ and $E'\subseteq\{aX=aX'\}$, both events by Basic Properties of Random Vectors: Coordinates, Borel Images and Arithmetic, Change of Variables, Almost Sure Equality and Pairs §almost-sure, and claim 2 of Basic Properties of a Measure gives $X+Y\sim_{P}X'+Y'$ and $aX\sim_{P}aX'$. The nonnegative random variables $\lVert X\rVert^{2}$ and $\lVert X'\rVert^{2}$ agree on $\{X=X'\}$, whose complement is an event of probability $0$, so they agree $P$-almost everywhere and $\mathbb{E}[\lVert X\rVert^{2}]=\mathbb{E}[\lVert X'\rVert^{2}]$ by The Lebesgue Integral and Null Sets: Almost-Everywhere Comparison, Markov's Inequality, and Dominated Convergence Almost Everywhere §comparison. If moreover $\mathbb{E}[\lVert X\rVert^{2}]<\infty$ and $\mathbb{E}[\lVert Y\rVert^{2}]<\infty$, then $X\cdot Y$ is integrable by claim 2 and agrees with $X'\cdot Y'$ on $E'$, hence almost everywhere, so $\mathbb{E}[X\cdot Y]=\mathbb{E}[X'\cdot Y']$ by the same clause.

Claim 4. By claim 2, $U\cdot W$, $V\cdot W$, $(U+V)\cdot W$ and $(aU)\cdot W$ are integrable ($U+V$ and $aU$ having finite second moment by claim 2). Pointwise, $(U+V)\cdot W=U\cdot W+V\cdot W$ by claim 2 of Bilinearity and Symmetry of the Dot Product on $\mathbb{R}^n$, $(aU)\cdot W=a\,(U\cdot W)$ by claim 4 there, and $U\cdot V=V\cdot U$ by claim 1 there; the first two identities pass to expectations by claim 2 of the integral theorem, and the third is immediate. By claim 2, $\mathbb{E}[U\cdot U]=\mathbb{E}[\lVert U\rVert^{2}]$, which is nonnegative as a nonnegative integral. By The Lebesgue Integral and Null Sets: Almost-Everywhere Comparison, Markov's Inequality, and Dominated Convergence Almost Everywhere §vanishing, $\mathbb{E}[\lVert U\rVert^{2}]=0$ if and only if $\lVert U\rVert^{2}=0$ almost everywhere. For each $\omega$, $\lVert U(\omega)\rVert^{2}=0$ holds exactly when $\lVert U(\omega)\rVert=0$ (claims 1 and 3 of Zero Products and Elementary Identities in a Field), exactly when $U(\omega)=0_{\mathbb{R}^{d}}$ (claim 3 of Elementary Properties of the Euclidean Norm on $\mathbb{R}^n$), that is, exactly when $\omega\in\{U=\mathbf{0}\}$. The set $\{U=\mathbf{0}\}$ is an event by Basic Properties of Random Vectors: Coordinates, Borel Images and Arithmetic, Change of Variables, Almost Sure Equality and Pairs §almost-sure; so $\lVert U\rVert^{2}=0$ almost everywhere exactly when $\Omega\setminus\{U=\mathbf{0}\}$ is contained in an event of probability $0$, which by claim 2 of Basic Properties of a Measure holds exactly when $P(\Omega\setminus\{U=\mathbf{0}\})=0$, that is, by claim 3 there, exactly when $P(U=\mathbf{0})=1$, which is $U\sim_{P}\mathbf{0}$.

Claim 5. By claim 3, if $X\sim_{P}X'$ and $Y\sim_{P}Y'$ with $X,Y\in\mathbf{L}^{2}$, then $X',Y'\in\mathbf{L}^{2}$ (as $\mathbb{E}[\lVert X'\rVert^{2}]=\mathbb{E}[\lVert X\rVert^{2}]$), $X+Y\sim_{P}X'+Y'$ and $aX\sim_{P}aX'$, and these sums and multiples lie in $\mathbf{L}^{2}$ by claim 2; so $[X+Y]=[X'+Y']$ and $[aX]=[aX']$ by claim 3, and the operations are well defined. Also $\mathbf{0}\in\mathbf{L}^{2}$, since $\lVert\mathbf{0}\rVert^{2}$ is the zero function by claim 3 of Elementary Properties of the Euclidean Norm on $\mathbb{R}^n$, whose integral is $0$ as noted in the first preliminary, and $(-1)X\in\mathbf{L}^{2}$ by claim 2. The axioms of Vector Space over a Field for $L^{2}$ are of two kinds. Those asserting identities (associativity and commutativity of addition, and the four axioms for scalar multiplication) concern elements built from classes by the two operations; through $[X]+[Y]=[X+Y]$ and $a[X]=[aX]$ each side is the class of a random vector built from representatives by the pointwise operations, and two random vectors that agree pointwise have the same class by reflexivity, so each such axiom reduces to the corresponding pointwise identity in $\mathbb{R}^{d}$, which holds by Euclidean Space $\mathbb{R}^n$ is a Real Vector Space. The two existence axioms are witnessed as follows: $[\mathbf{0}]$ is a zero vector because $X+\mathbf{0}=X$ pointwise, the origin being the zero vector of $\mathbb{R}^{d}$ by Euclidean Space $\mathbb{R}^n$ is a Real Vector Space; and $[(-1)X]$ is an additive inverse of $[X]$ because $X+(-1)X=\mathbf{0}$ pointwise, by claim 5 of Elementary Identities in a Vector Space and the inverse axiom in $\mathbb{R}^{d}$. Hence $L^{2}$ is a real vector space with zero vector $[\mathbf{0}]$ and $-[X]=[(-1)X]$.