Proof of Independent Jointly Gaussian Families are Jointly Gaussian

Throughout, $\delta_{ab}$ denotes $1$ when $a=b$ and $0$ otherwise, and $I_k$ is the identity matrix.

Step 0 (cross-group covariances). Let $q\ne q'$, $j\in J_q$, $j'\in J_{q'}$. The events $(X^{q}_{j})^{-1}(B)$ lie in $\sigma(X^{q}_{i}:i\in J_q)$ and the events $(X^{q'}_{j'})^{-1}(B')$ lie in $\sigma(X^{q'}_{i}:i\in J_{q'})$ for all Borel sets $B,B'$, so by independence of these $\sigma$-algebras and the closing remark of Sigma-Algebra Generated by Random Variables and Independence of Sigma-Algebras, $X^{q}_{j}$ and $X^{q'}_{j'}$ are independent. Each is a Gaussian random variable (one-member subfamily, Jointly Gaussian Families of Random Variables and Gaussian Processes), hence square-integrable by Square-Integrability, Moments, and Covariance Matrix of a Gaussian Random Vector, and by Expectation of a Product of Independent Random Variables the expectation of the product is the product of expectations, so $\operatorname{Cov}(X^{q}_{j},X^{q'}_{j'})=0$ with the covariance.

Step 1 (canonical representation of a Gaussian random vector). Let $(Y_1,\dots,Y_d)$ be a Gaussian random vector with expectations $\mu_i=\mathbb{E}[Y_i]$ (finite by Square-Integrability, Moments, and Covariance Matrix of a Gaussian Random Vector), and put $\widehat{Y}_i=Y_i-\mu_i$; the tuple $(\widehat{Y}_1,\dots,\widehat{Y}_d)$ is a Gaussian random vector by Affine Transformations of Gaussian Random Vectors are Gaussian, and $\mathbb{E}[\widehat{Y}_i]=0$. We construct $k\ge0$ and random variables $\zeta_1,\dots,\zeta_k$, each a finite linear combination of $\widehat{Y}_1,\dots,\widehat{Y}_d$, such that $\zeta_1,\dots,\zeta_k$ are independent standard normal random variables and each $\widehat{Y}_i$ is almost surely equal to a linear combination of $\zeta_1,\dots,\zeta_k$.

Call a subset $\{i_1,\dots,i_k\}\subseteq\{1,\dots,d\}$ admissible if it is empty or the Gram matrix $G=(\mathbb{E}[\widehat{Y}_{i_a}\widehat{Y}_{i_b}])_{1\le a,b\le k}$ is symmetric positive definite in the sense of Triangular Orthonormalization of a Positive Definite Gram Matrix. The empty set is admissible, so there is an admissible subset that is maximal with respect to inclusion; fix one, say $\{i_1,\dots,i_k\}$ with $k\ge0$.

If $k=0$, take the family $(\zeta_a)$ empty and proceed directly to the residual paragraph below. If $k\ge1$: by claim 1 of Triangular Orthonormalization of a Positive Definite Gram Matrix there is an invertible lower triangular $T$ with $TGT^{\top}=I_k$; set $\zeta_a=\sum_{b\le a}T_{ab}\widehat{Y}_{i_b}$. Then $\mathbb{E}[\zeta_a]=\sum_{b\le a}T_{ab}\,\mathbb{E}[\widehat{Y}_{i_b}]=0$ and $\mathbb{E}[\zeta_a\zeta_b]=(TGT^{\top})_{ab}=\delta_{ab}$, so the $\zeta_a$ are centered, pairwise uncorrelated, and each has variance $\operatorname{Var}(\zeta_a)=\mathbb{E}[\zeta_a^2]=1>0$. They are jointly Gaussian by Affine Transformations of Gaussian Random Vectors are Gaussian, hence independent by Pairwise Uncorrelated Jointly Gaussian Random Variables are Independent; and applying the standardization claim of Standardization and Cumulative Distribution Function of a Gaussian Random Variable with mean $0$ and standard deviation $1$ shows each $\zeta_a=(\zeta_a-0)/1$ has the standard normal distribution, i.e. is a standard normal random variable. By claim 2 of Triangular Orthonormalization of a Positive Definite Gram Matrix, $T^{-1}$ exists, and $\widehat{Y}_{i_a}=\sum_b(T^{-1})_{ab}\zeta_b$ exactly (as an identity of random variables).

Residual. Let $i\notin\{i_1,\dots,i_k\}$ (if any). Put $R=\widehat{Y}_i-\sum_{a=1}^{k}\mathbb{E}[\widehat{Y}_i\zeta_a]\,\zeta_a$ (for $k=0$, $R=\widehat{Y}_i$); then $\mathbb{E}[R\,\zeta_a]=0$ for every $a$. We claim $\mathbb{E}[R^2]=0$. Suppose not. For arbitrary reals $\alpha_1,\dots,\alpha_k,\beta$, the identity $\widehat{Y}_{i_a}=\sum_b(T^{-1})_{ab}\zeta_b$ and the definition of $R$ give

and by $\mathbb{E}[\zeta_a\zeta_b]=\delta_{ab}$ and $\mathbb{E}[R\zeta_b]=0$,

If this vanishes then $\beta=0$ (because $\mathbb{E}[R^2]>0$) and every $\gamma_b=0$; with $\beta=0$ this reads $\sum_a\alpha_a(T^{-1})_{ab}=0$ for all $b$, and invertibility of $T^{-1}$ forces $\alpha=0$. Hence the Gram matrix of $(\widehat{Y}_{i_1},\dots,\widehat{Y}_{i_k},\widehat{Y}_{i})$ is positive definite (it is symmetric since $\mathbb{E}[\widehat{Y}\widehat{Y}']$ is symmetric in its arguments), so $\{i_1,\dots,i_k,i\}$ is admissible, contradicting maximality. Therefore $\mathbb{E}[R^2]=0$, and by the null-equivalence of Square-Integrable Random Variables and the Mean-Square Inner Product, $\widehat{Y}_i=\sum_a\mathbb{E}[\widehat{Y}_i\zeta_a]\zeta_a$ almost surely. Together with the exact identities for the $\widehat{Y}_{i_a}$, every $\widehat{Y}_i$ ($1\le i\le d$) is almost surely a linear combination of $\zeta_1,\dots,\zeta_k$ (for $k=0$: almost surely $0$, the empty combination).

Step 2 (finite subfamilies of the combined family). Let finitely many distinct index pairs $(q,j)$ of the combined family be given, say involving groups $q=1,\dots,n$ (groups not represented are simply omitted below). For each represented $q$, the members selected from group $q$ form a Gaussian random vector $Y^{q}$ (a finite subfamily of a jointly Gaussian family, Jointly Gaussian Families of Random Variables and Gaussian Processes); apply Step 1 to obtain $\zeta^{q}_1,\dots,\zeta^{q}_{k_q}$. Each $\zeta^{q}_a$ is a linear combination of constant shifts of selected members of group $q$; such combinations are measurable with respect to $\mathcal{A}_q=\sigma(X^{q}_{j}:j\in J_q)$. Indeed, for $\mathcal{A}_q$-measurable $X$, real $c\ne0$ and real $\mu$, the set where $c(X-\mu)>\lambda$ is the set where $X>\mu+\lambda/c$ or $X<\mu+\lambda/c$ according to the sign of $c$, hence lies in $\mathcal{A}_q$; for $\mathcal{A}_q$-measurable $X,Y$, the set where $X+Y>\lambda$ is the countable union over rational $r$ of the intersections of the sets where $X>r$ and where $Y>\lambda-r$, hence lies in $\mathcal{A}_q$; and the generator criterion, applied on the measurable space $(\Omega,\mathcal{A}_q)$, upgrades these level-set memberships to $\mathcal{A}_q$-measurability.

The full collection $\{\zeta^{q}_a:q\ \text{represented},\ 1\le a\le k_q\}$ is independent: for events $A^{q}_a=(\zeta^{q}_a)^{-1}(B^{q}_a)$ with Borel $B^{q}_a$,

where the first equality holds because $\bigcap_a A^{q}_a\in\mathcal{A}_q$ and the $\mathcal{A}_q$ are independent (Sigma-Algebra Generated by Random Variables and Independence of Sigma-Algebras), and the second holds by within-group independence from Step 1; subfamilies are covered by inserting $\Omega$ for omitted factors, exactly as in the closing remark of Sigma-Algebra Generated by Random Variables and Independence of Sigma-Algebras. By Independence of Events and of Random Variables this is independence of all the $\zeta^{q}_a$; each is standard normal by Step 1.

Finally, each selected member equals, almost surely, its expectation plus a linear combination of the $\zeta^{q}_a$ of its own group (Step 1). Thus the tuple of selected members admits a Gaussian representation in the sense of Gaussian Random Vectors and Jointly Gaussian Random Variables, with the combined independent standard normal family $(\zeta^{q}_a)_{q,a}$ (or $m=0$ if all groups produced empty families); hence it is a Gaussian random vector. Since the finite choice of distinct index pairs was arbitrary, the combined family is jointly Gaussian by Jointly Gaussian Families of Random Variables and Gaussian Processes. Together with Step 0, this proves the lemma. $\blacksquare$