Proof of Gaussian Structure and Moment Equations for Linear Stochastic Differential Equations

Throughout, a real-valued function on a subinterval $I$ of the real numbers $\mathbb{R}$ is called continuous on $I$ when it is continuous relative to $I$, both $I$ and the codomain $\mathbb{R}$ carrying the metric of the real line.

Claim 1. Write $\mathcal{B}$ for the base family of all $\xi^{i}$ and all $W^{j}_t$, jointly Gaussian by hypothesis. Every member of the enlarged family is a mean-square limit of finite linear — hence affine — combinations of members of $\mathcal{B}$: for the Wiener integrals this holds by claim 1 of Adapted Mean-Square Continuous Processes are Ito Integrable together with claim 1 of Existence and Uniqueness of the Mean-Square Extension of the Elementary Stochastic Integral (the elementary stochastic integrals of the deterministic grid-value approximants are finite linear combinations of increments of the $W^{j}$), and for the $X^{i}_t$ it is claim 3 of Existence, Uniqueness, and Variation of Constants for Linear Stochastic Differential Equations (with the forcing values absent, since every $g_r$ is the zero tuple). By Mean-Square Limits of Affine Combinations Adjoin to a Jointly Gaussian Family, the enlarged family is jointly Gaussian.

Claim 2. Taking expectations in the defining equation of Mean-Square Solution of a Linear Stochastic Differential Equation with Additive Wiener Noise (with vanishing forcing): Wiener integrals are centered (claim 2 of Wiener Integrals Against a Vector Brownian Motion are Jointly Gaussian), and by claim 3 of Basic Properties of the Mean-Square Riemann Integral (with $Z=1$) together with linearity of the expectation and of the Riemann integral (claim 3 of Restricted Lebesgue Measure and Integral Toolkit on a Compact Interval, Linearity and Monotonicity of the Lebesgue Integral),

with $\mu^{i}$ continuous (claim 3 of Basic Properties of the Mean-Square Riemann Integral). This is the linear system of claim 3 of Fundamental Solution and Variation of Constants for Linear Ordinary Differential Equations with vanishing inhomogeneity on $[0,T]$, whose unique continuous solution is $\Phi(t)\,\mathbb{E}[\xi]$.

Claim 3. Fix $i,k$. The pair $(Y,Z)=(X^{i},X^{k})$ is of the form treated in Second-Moment Evolution for Processes of Integral Form on $[0,T]$: $X^{i}_t=\xi^{i}+\int_0^t\alpha_r\,dr+\sum_j\int_0^t\varepsilon_{ij}(r)\,dW^{j}_r$ with $\alpha_r=\sum_jA_{ij}(r)X^{j}_r$, mean-square continuous by claims 1-2 of Basic Properties of the Mean-Square Riemann Integral (and similarly for $X^{k}$). The orthogonality hypothesis of that lemma holds by its claim 2: for every $s$, $X^{i}_s$ and $X^{k}_s$ lie in the closed mean-square span of the $\xi^{i'}$ and the $W^{j'}_r$ with $r\le s$ (claim 3 of Existence, Uniqueness, and Variation of Constants for Linear Stochastic Differential Equations), and $\sigma(\xi^{1},\dots,\xi^{l})$ is independent of the $W$-$\sigma$-algebra by hypothesis. Hence, by claim 1 of Second-Moment Evolution for Processes of Integral Form,

The functions $\mu^{i}$ are continuous on $[0,T]$ and, by claim 3 of Fundamental Theorem of Calculus, Part I, on a Closed Real Interval applied to claim 2's integral equation (whose integrand $r\mapsto\sum_jA_{ij}(r)\mu^{j}(r)$ is continuous on $[0,T]$ by claim 5 of Continuity of Sums and Products of Real-Valued Functions on a Metric Space), differentiable at every point of $(0,T)$ with $(\mu^{i})'=\sum_jA_{ij}\mu^{j}$. By the product rule, claim 3 of Sum, Constant Multiple, and Product Rules for One-Dimensional Derivatives, the function $\mu^{i}\mu^{k}$ is differentiable at every point of $(0,T)$ with derivative $q:=\sum_jA_{ij}\mu^{j}\mu^{k}+\mu^{i}\sum_jA_{kj}\mu^{j}$, and $\mu^{i}\mu^{k}$ is continuous on $[0,T]$ by claim 5 of Continuity of Sums and Products of Real-Valued Functions on a Metric Space. The function $q$ is likewise continuous on $[0,T]$ by claim 5 of Continuity of Sums and Products of Real-Valued Functions on a Metric Space, hence Riemann integrable on $[0,T]$ by claim 3 of Restricted Lebesgue Measure and Integral Toolkit on a Compact Interval. Now fix $t\in(0,T]$. By claim 1 of Restriction Stability of Continuity and of the Derivative the restrictions of $\mu^{i}\mu^{k}$ and of $q$ to $[0,t]$ are continuous on $[0,t]$, so the restriction of $q$ is Riemann integrable on $[0,t]$ by claim 3 of Restricted Lebesgue Measure and Integral Toolkit on a Compact Interval; and by claim 2 of Restriction Stability of Continuity and of the Derivative the restriction of $\mu^{i}\mu^{k}$ is differentiable at every point of $(0,t)$ with derivative $q$ there. Hence Fundamental Theorem of Calculus, Part II, on a Closed Real Interval, applied on $[0,t]$, gives the corresponding integral identity for that $t$; the degenerate case $t=0$ holds by the convention $\int_0^0:=0$. Subtracting it from the display above and using $\operatorname{Cov}(X^{i}_t,X^{k}_t)=\mathbb{E}[X^{i}_tX^{k}_t]-\mu^{i}(t)\mu^{k}(t)$ (Covariance of Square-Integrable Random Variables) and linearity of the Riemann integral,

with $P_{ik}(0)=\mathbb{E}[\xi^{i}\xi^{k}]-\mathbb{E}[\xi^{i}]\mathbb{E}[\xi^{k}]=\operatorname{Cov}(\xi^{i},\xi^{k})$. Since $P_{ij}=P_{ji}$ (symmetry of the covariance), $\sum_jA_{kj}P_{ij}=(PA^{\top})_{ik}$ by Product of Real Matrices and Transpose of a Real Matrix, and $\sum_j\varepsilon_{ij}\varepsilon_{kj}=(\varepsilon\varepsilon^{\top})_{ik}$, this is exactly the asserted entrywise equation. Its entries are continuous (claim 1 of Second-Moment Evolution for Processes of Integral Form and continuity of $\mu^{i}\mu^{k}$), so $P$ is a continuous solution of the equation of claim 1 of Lyapunov Representation and Positive Semidefiniteness for Linear Matrix Equations for the data $(A,\varepsilon\varepsilon^{\top},P_0)$, hence the unique one. $\blacksquare$