Proof of The Law of a Gaussian Random Vector Is Determined by Its Mean and Covariance Matrix

Throughout, means and covariance matrices of affine functions of square-integrable random vectors are computed by Rules for Expectations, Variances and Covariances (vi) and Rules for Expectations, Variances and Covariances (xiii); Borel functions of random vectors are random vectors with the image laws, and almost surely equal random vectors have the same law (Random Vectors: Coordinates, Functions, Pairs and Almost-Sure Equality (ii), Random Vectors: Coordinates, Functions, Pairs and Almost-Sure Equality (iv)). We use repeatedly that a real matrix $D$ with $DD^{\top}=0$ vanishes, since $\|D^{\top}y\|^{2}=y\cdot DD^{\top}y=0$ for every $y$. In (i), $M=(B^{\top}B)^{-1}B^{\top}\in\mathbb{R}^{r\times d}$, so that $MB=I_{r}$ and

$$BM\Sigma=BMBB^{\top}=BB^{\top}=\Sigma,\qquad M\Sigma M^{\top}=MBB^{\top}M^{\top}=I_{r}.$$

Step 1 ((i), the case $\Sigma=0_{d}$). We split on Gaussian Random Vectors and Jointly Gaussian Random Variables (i). If $X=m_{0}$ almost surely for some $m_{0}\in\mathbb{R}^{d}$, then $m_{0}=\mathbb{E}X=m$ by Square-Integrability, Mean and Covariance Matrix of a Gaussian Random Vector (iii). Otherwise $X$ has a Gaussian representation $(k,Z,m',A)$, and $m'=m$ and $AA^{\top}=\Sigma=0_{d}$ by Square-Integrability, Mean and Covariance Matrix of a Gaussian Random Vector (ii); so $A=0$, and $X=m'+AZ=m$ on the event $\{X=m'+AZ\}$, which has probability $1$.

Step 2 ((i), $W$ is standard normal). $X$ is not almost surely constant: otherwise $\Sigma=0_{d}$ (Square-Integrability, Mean and Covariance Matrix of a Gaussian Random Vector (iii)), so $BB^{\top}=0_{d}$ gives $B=0$, and $MBe_{1}=0\neq e_{1}$, against $MB=I_{r}$. So $X$ has a Gaussian representation $(k,Z,m',A)$ (Gaussian Random Vectors and Jointly Gaussian Random Variables (i)), and $m'=m$, $AA^{\top}=\Sigma$ by Square-Integrability, Mean and Covariance Matrix of a Gaussian Random Vector (ii). On the event $\Omega_{0}=\{X=m+AZ\}$, which has probability $1$, $W=CZ$ with $C=MA\in\mathbb{R}^{r\times k}$, and $CC^{\top}=M\Sigma M^{\top}=I_{r}$. By Standard Normal Vectors: Density, Moments, Orthogonal Invariance and Concatenation (vi), $CZ$ is a standard normal vector; as $W=CZ$ almost surely, $P_{W}=P_{CZ}$, and $W$ is a standard normal vector by Standard Normal Vectors: Density, Moments, Orthogonal Invariance and Concatenation (iii).

Step 3 ((i), $X=m+BW$ almost surely). With $D=(I_{d}-BM)A$, expanding $DD^{\top}=(I_{d}-BM)AA^{\top}(I_{d}-BM)^{\top}$ with $AA^{\top}=\Sigma$, $\Sigma^{\top}=\Sigma$, $BM\Sigma=\Sigma$ and $M\Sigma M^{\top}=I_{r}$ (so that $\Sigma(BM)^{\top}=(BM\Sigma)^{\top}=\Sigma$ and $BM\Sigma(BM)^{\top}=BB^{\top}=\Sigma$) gives

$$DD^{\top}=(I_{d}-BM)\Sigma(I_{d}-BM)^{\top}=0_{d},$$

so $D=0$, that is, $BMA=A$. Hence, on the event $\Omega_{0}$ of Step 2, $BW=BMAZ=AZ$ and

$$X=m+AZ=m+BW .$$

Step 4 ((ii)). Let $m$ and $\Sigma$ be the common mean and covariance matrix; $\Sigma$ is positive semidefinite (Rules for Expectations, Variances and Covariances (xii)). If $\Sigma=0_{d}$, then $X=m$ and $X'=m$ almost surely (Step 1, on each probability space), so for every Borel set $U\subseteq\mathbb{R}^{d}$ both $P_{X}(U)$ and $P'_{X'}(U)$ equal $1$ if $m\in U$ and $0$ otherwise. Otherwise fix $r$ and $B$ by A Nonzero Positive Semidefinite Matrix Factors as BBᵀ with BᵀB Invertible (i). By (i), $X=m+BW$ and $X'=m+BW'$ almost surely, with $W$ and $W'$ standard normal vectors in $\mathbb{R}^{r}$. Let $\gamma_{r}$ be the standard normal distribution on $\mathbb{R}^{r}$ and $T(w)=m+Bw$ (a Borel map, being a Gaussian random vector on $(\mathbb{R}^{r},\mathcal{B}(\mathbb{R}^{r}),\gamma_{r})$ as the affine image of the identity, which is standard normal and hence Gaussian by Standard Normal Vectors: Density, Moments, Orthogonal Invariance and Concatenation (ii), Affine Transformations of Gaussian Random Vectors are Gaussian (iii) and Affine Transformations of Gaussian Random Vectors are Gaussian (i)). Now Standard Normal Vectors: Density, Moments, Orthogonal Invariance and Concatenation (iii) gives

$$P_{X}=P_{W}\circ T^{-1}=\gamma_{r}\circ T^{-1}=P'_{W'}\circ T^{-1}=P'_{X'}.$$

Step 5 ((iii)). By Standard Normal Vectors: Density, Moments, Orthogonal Invariance and Concatenation (ii), the identity $Z$ of the probability space $(\Omega',\mathcal{F}',P')=(\mathbb{R}^{d},\mathcal{B}(\mathbb{R}^{d}),\gamma_{d})$ is a standard normal vector in $\mathbb{R}^{d}$. Let $X'=m+\Sigma^{1/2}Z$, where $\Sigma^{1/2}\in\mathcal{S}(d)$ and $\Sigma^{1/2}\Sigma^{1/2}=\Sigma$ (The Positive Semidefinite Square Root of a Positive Semidefinite Matrix (i)). Then $(d,Z,m,\Sigma^{1/2})$ is a Gaussian representation of $X'$, so $X'$ is Gaussian and, by Square-Integrability, Mean and Covariance Matrix of a Gaussian Random Vector (ii), $\mathbb{E}X'=m$ and $\operatorname{Cov}(X')=\Sigma^{1/2}(\Sigma^{1/2})^{\top}=\Sigma$.