Proof of Square-Integrability, Moments, and Covariance Matrix of a Gaussian Random Vector

Throughout, write $W_i=\mu_i+\sum_{j=1}^{m}a_{ij}Z_j$ and $D_i=X_i-W_i$; both are random variables by the closure of random variables under sums and scalar multiples recorded in the preliminaries of Square-Integrable Random Variables and the Mean-Square Inner Product, and $P(D_i=0)=1$ by the definition of a Gaussian representation. When $m=0$, every sum over $j$ below is empty and equal to $0$, in accordance with Gaussian Random Vectors and Jointly Gaussian Random Variables, and every assertion about the individual $Z_j$ holds vacuously; in that case $W_i=\mu_i$ is a constant random variable. All norms and inner products are those of Square-Integrable Random Variables and the Mean-Square Inner Product.

Claim 1. Each $Z_j$ is square-integrable, since $\mathbb{E}[Z_j^{2}]=1$ by Claim 1 of Moments and Stability of the Standard Normal Distribution; hence each $W_i$ is square-integrable by the closure properties of Square-Integrable Random Variables and the Mean-Square Inner Product (constants being square-integrable). Next, $D_i^{2}$ is a nonnegative random variable that vanishes off the event $N_i=\{X_i\ne W_i\}$, which satisfies $P(N_i)=0$. If $s$ is a simple function with $0\le s\le D_i^{2}$ pointwise, then $s$ vanishes wherever $D_i^{2}$ does, so every set on which $s$ takes a nonzero value is a subset of $N_i$ and has probability $0$ by the monotonicity of the measure $P$; hence the integral of $s$ is $0$. Taking the supremum over such $s$ in Lebesgue Integral of a Nonnegative Measurable Function gives $\mathbb{E}[D_i^{2}]=0$. Thus $D_i$ is square-integrable with $\lVert D_i\rVert_{2}=0$, and $X_i=W_i+D_i$ is square-integrable by closure. Moreover, for every square-integrable $Y$, the Cauchy-Schwarz inequality gives $|\mathbb{E}[D_iY]|\le\lVert D_i\rVert_{2}\lVert Y\rVert_{2}=0$, so by the linearity of expectation from Linearity and Monotonicity of the Lebesgue Integral,

$$\mathbb{E}[X_iY]=\mathbb{E}[W_iY]\qquad\text{for every square-integrable }Y. \tag{$*$}$$

Claim 2. Taking $Y$ the constant random variable $1$ in the identity of Claim 1 and using linearity of expectation together with $\mathbb{E}[Z_j]=0$ (Claim 1 of Moments and Stability of the Standard Normal Distribution),

$$\mathbb{E}[X_i]=\mathbb{E}[W_i]=\mu_i+\sum_{j=1}^{m}a_{ij}\,\mathbb{E}[Z_j]=\mu_i .$$

For the covariance, note $X_i-\mu_i=\sum_{j}a_{ij}Z_j+D_i$, and the terms involving $D_i$ or $D_k$ contribute $0$ to every expectation of a product against a square-integrable factor, by the Cauchy-Schwarz bound of Claim 1. Expanding with the bilinearity of the mean-square inner product from Square-Integrable Random Variables and the Mean-Square Inner Product,

$$\operatorname{Cov}(X_i,X_k)=\mathbb{E}\Bigl[\Bigl(\sum_{j=1}^{m}a_{ij}Z_j\Bigr)\Bigl(\sum_{l=1}^{m}a_{kl}Z_l\Bigr)\Bigr]=\sum_{j=1}^{m}\sum_{l=1}^{m}a_{ij}\,a_{kl}\,\mathbb{E}[Z_jZ_l].$$

For $j\ne l$, the pair $Z_j,Z_l$ is independent (a pair from an independent family is independent, directly from that definition), and both are integrable, so $\mathbb{E}[Z_jZ_l]=\mathbb{E}[Z_j]\,\mathbb{E}[Z_l]=0$ by Expectation of a Product of Independent Random Variables; and $\mathbb{E}[Z_j^{2}]=1$. Hence $\operatorname{Cov}(X_i,X_k)=\sum_{j=1}^{m}a_{ij}a_{kj}$.

Claim 3. The mean vector and covariance matrix are defined from $X_1,\dots,X_d$ alone, so they are representation-independent, and Claim 2 evaluates them in any representation. Symmetry follows from the formula of Claim 2 and commutativity of real multiplication. For positive semidefiniteness, exchanging the order of the finite sums,

$$\sum_{i=1}^{d}\sum_{k=1}^{d}c_i\,c_k\,\Sigma_{ik}=\sum_{j=1}^{m}\Bigl(\sum_{i=1}^{d}c_i\,a_{ij}\Bigr)\Bigl(\sum_{k=1}^{d}c_k\,a_{kj}\Bigr)=\sum_{j=1}^{m}\Bigl(\sum_{i=1}^{d}c_i\,a_{ij}\Bigr)^{2}\ge0 .$$

Claim 4. Applying Existence of Independent Sequences with Prescribed Distributions with every prescribed distribution equal to the standard normal distribution yields a probability space carrying an independent sequence of standard normal random variables. Given independent standard normals $Z_1,\dots,Z_m$ on any probability space and real numbers $(\mu_i)$, $(a_{ij})$, the functions $X_i=\mu_i+\sum_{j}a_{ij}Z_j$ are random variables by the closure preliminaries of Square-Integrable Random Variables and the Mean-Square Inner Product, and the defining equalities hold at every point of $\Omega$, hence with probability $1$; so $\bigl(m,(\mu_i),(a_{ij}),(Z_j)\bigr)$ is a Gaussian representation of $(X_1,\dots,X_d)$ in the sense of Gaussian Random Vectors and Jointly Gaussian Random Variables. $\blacksquare$