Proof of Gaussian Structure and Moment Equations for Linear Stochastic Differential Equations

Claim 1. Write $\mathcal{B}$ for the base family of all $\xi^{i}$ and all $W^{j}_t$, jointly Gaussian by hypothesis. Every member of the enlarged family is a mean-square limit of finite linear — hence affine — combinations of members of $\mathcal{B}$: for the Wiener integrals this holds by claim 1 of Adapted Mean-Square Continuous Processes are Ito Integrable together with claim 1 of Existence and Uniqueness of the Mean-Square Extension of the Elementary Stochastic Integral (the elementary stochastic integrals of the deterministic grid-value approximants are finite linear combinations of increments of the $W^{j}$), and for the $X^{i}_t$ it is claim 3 of Existence, Uniqueness, and Variation of Constants for Linear Stochastic Differential Equations (with the forcing values absent, since every $g_r$ is the zero tuple). By Mean-Square Limits of Affine Combinations Adjoin to a Jointly Gaussian Family, the enlarged family is jointly Gaussian.

Claim 2. Taking expectations in the defining equation of Mean-Square Solution of a Linear Stochastic Differential Equation with Additive Wiener Noise (with vanishing forcing): Wiener integrals are centered (claim 2 of Wiener Integrals Against a Vector Brownian Motion are Jointly Gaussian), and by claim 3 of Basic Properties of the Mean-Square Riemann Integral (with $Z=1$) together with linearity of the expectation and of the Riemann integral (Agreement of the Riemann and Lebesgue Integrals for Continuous Functions on a Closed Interval, Linearity and Monotonicity of the Lebesgue Integral),

with $\mu^{i}$ continuous (claim 3 of Basic Properties of the Mean-Square Riemann Integral). This is the linear system of claim 3 of Fundamental Solution and Variation of Constants for Linear Ordinary Differential Equations with vanishing inhomogeneity on $[0,T]$, whose unique continuous solution is $\Phi(t)\,\mathbb{E}[\xi]$.

Claim 3. Fix $i,k$. The pair $(Y,Z)=(X^{i},X^{k})$ is of the form treated in Second-Moment Evolution for Processes of Integral Form on $[0,T]$: $X^{i}_t=\xi^{i}+\int_0^t\alpha_r\,dr+\sum_j\int_0^t\varepsilon_{ij}(r)\,dW^{j}_r$ with $\alpha_r=\sum_jA_{ij}(r)X^{j}_r$, mean-square continuous by claims 1-2 of Basic Properties of the Mean-Square Riemann Integral (and similarly for $X^{k}$). The orthogonality hypothesis of that lemma holds by its claim 2: for every $s$, $X^{i}_s$ and $X^{k}_s$ lie in the closed mean-square span of the $\xi^{i'}$ and the $W^{j'}_r$ with $r\le s$ (claim 3 of Existence, Uniqueness, and Variation of Constants for Linear Stochastic Differential Equations), and $\sigma(\xi^{1},\dots,\xi^{l})$ is independent of the $W$-$\sigma$-algebra by hypothesis. Hence, by claim 1 of Second-Moment Evolution for Processes of Integral Form,

The functions $\mu^{i}$ are continuous and, by Fundamental Theorem of Calculus, Part I in One Dimension applied to claim 2's integral equation, differentiable at interior points with $(\mu^{i})'=\sum_jA_{ij}\mu^{j}$; the product rule (Sum and Product Rules for One-Dimensional Derivatives and Continuity) gives that $\mu^{i}\mu^{k}$ is continuous on $[0,T]$, differentiable at interior points with derivative $\sum_jA_{ij}\mu^{j}\mu^{k}+\mu^{i}\sum_jA_{kj}\mu^{j}$, hence an antiderivative of that continuous function, and Fundamental Theorem of Calculus, Part II in One Dimension gives the corresponding integral identity. Subtracting it from the display above and using $\operatorname{Cov}(X^{i}_t,X^{k}_t)=\mathbb{E}[X^{i}_tX^{k}_t]-\mu^{i}(t)\mu^{k}(t)$ (Covariance of Square-Integrable Random Variables) and linearity of the Riemann integral,

with $P_{ik}(0)=\mathbb{E}[\xi^{i}\xi^{k}]-\mathbb{E}[\xi^{i}]\mathbb{E}[\xi^{k}]=\operatorname{Cov}(\xi^{i},\xi^{k})$. Since $P_{ij}=P_{ji}$ (symmetry of the covariance), $\sum_jA_{kj}P_{ij}=(PA^{\top})_{ik}$ by Product of Real Matrices and Transpose of a Real Matrix, and $\sum_j\varepsilon_{ij}\varepsilon_{kj}=(\varepsilon\varepsilon^{\top})_{ik}$, this is exactly the asserted entrywise equation. Its entries are continuous (claim 1 of Second-Moment Evolution for Processes of Integral Form and continuity of $\mu^{i}\mu^{k}$), so $P$ is a continuous solution of the equation of claim 1 of Lyapunov Representation and Positive Semidefiniteness for Linear Matrix Equations for the data $(A,\varepsilon\varepsilon^{\top},P_0)$, hence the unique one. $\blacksquare$