Proof of Square-Integrability, Moments, and Covariance Matrix of a Gaussian Random Vector

Throughout, write $W_i=\mu_i+\sum_{j=1}^{m}a_{ij}Z_j$ and $D_i=X_i-W_i$; both are random variables by the closure of random variables under sums and scalar multiples recorded in the preliminaries of Square-Integrable Random Variables and the Mean-Square Inner Product, and $P(D_i=0)=1$ by the definition of a Gaussian representation. When $m=0$, every sum over $j$ below is empty and equal to $0$, in accordance with Gaussian Random Vectors and Jointly Gaussian Random Variables, and every assertion about the individual $Z_j$ holds vacuously; in that case $W_i=\mu_i$ is a constant random variable. All norms and inner products are those of Square-Integrable Random Variables and the Mean-Square Inner Product.

Claim 1. Each $Z_j$ is square-integrable, since $\mathbb{E}[Z_j^{2}]=1$ by Claim 1 of Moments and Stability of the Standard Normal Distribution; hence each $W_i$ is square-integrable by the closure properties of Square-Integrable Random Variables and the Mean-Square Inner Product (constants being square-integrable). Next, $D_i^{2}$ is a nonnegative random variable that vanishes off the event $N_i=\{X_i\ne W_i\}$, which satisfies $P(N_i)=0$. If $s$ is a simple function with $0\le s\le D_i^{2}$ pointwise, then $s$ vanishes wherever $D_i^{2}$ does, so every set on which $s$ takes a nonzero value is a subset of $N_i$ and has probability $0$ by the monotonicity of the measure $P$; hence the integral of $s$ is $0$. Taking the supremum over such $s$ in Lebesgue Integral of a Nonnegative Measurable Function gives $\mathbb{E}[D_i^{2}]=0$. Thus $D_i$ is square-integrable with $\lVert D_i\rVert_{2}=0$, and $X_i=W_i+D_i$ is square-integrable by closure. Moreover, for every square-integrable $Y$, the Cauchy-Schwarz inequality gives $|\mathbb{E}[D_iY]|\le\lVert D_i\rVert_{2}\lVert Y\rVert_{2}=0$, so by the linearity of expectation from Linearity and Monotonicity of the Lebesgue Integral,

Claim 2. Taking $Y$ the constant random variable $1$ in the identity of Claim 1 and using linearity of expectation together with $\mathbb{E}[Z_j]=0$ (Claim 1 of Moments and Stability of the Standard Normal Distribution),

For the covariance, note $X_i-\mu_i=\sum_{j}a_{ij}Z_j+D_i$, and the terms involving $D_i$ or $D_k$ contribute $0$ to every expectation of a product against a square-integrable factor, by the Cauchy-Schwarz bound of Claim 1. Expanding with the bilinearity of the mean-square inner product from Square-Integrable Random Variables and the Mean-Square Inner Product,

For $j\ne l$, the pair $Z_j,Z_l$ is independent (a pair from an independent family is independent, directly from that definition), and both are integrable, so $\mathbb{E}[Z_jZ_l]=\mathbb{E}[Z_j]\,\mathbb{E}[Z_l]=0$ by Expectation of a Product of Independent Random Variables; and $\mathbb{E}[Z_j^{2}]=1$. Hence $\operatorname{Cov}(X_i,X_k)=\sum_{j=1}^{m}a_{ij}a_{kj}$.

Claim 3. The mean vector and covariance matrix are defined from $X_1,\dots,X_d$ alone, so they are representation-independent, and Claim 2 evaluates them in any representation. Symmetry follows from the formula of Claim 2 and commutativity of real multiplication. For positive semidefiniteness, exchanging the order of the finite sums,

Claim 4. Applying Existence of Independent Sequences with Prescribed Distributions with every prescribed distribution equal to the standard normal distribution yields a probability space carrying an independent sequence of standard normal random variables. Given independent standard normals $Z_1,\dots,Z_m$ on any probability space and real numbers $(\mu_i)$, $(a_{ij})$, the functions $X_i=\mu_i+\sum_{j}a_{ij}Z_j$ are random variables by the closure preliminaries of Square-Integrable Random Variables and the Mean-Square Inner Product, and the defining equalities hold at every point of $\Omega$, hence with probability $1$; so $\bigl(m,(\mu_i),(a_{ij}),(Z_j)\bigr)$ is a Gaussian representation of $(X_1,\dots,X_d)$ in the sense of Gaussian Random Vectors and Jointly Gaussian Random Variables. $\blacksquare$