Proof of Second-Moment Evolution for Processes of Integral Form

Throughout, $\lVert\cdot\rVert_2$, Cauchy-Schwarz, and the triangle inequality are from Square-Integrable Random Variables and the Mean-Square Inner Product and Cauchy-Schwarz and Triangle Inequalities for the Mean-Square Norm; bilinearity of the covariance and the identity $\operatorname{Cov}(V,V')=\mathbb{E}[VV']-\mathbb{E}[V]\mathbb{E}[V']$ are from that definition and the bilinearity of the mean-square inner product; Wiener integrals are centered with the covariances of claim 3 of Wiener Integrals Against a Vector Brownian Motion are Jointly Gaussian. Write $\Delta^{f}_j(s,t)=\int_a^tf_j\,dW^{j}-\int_a^sf_j\,dW^{j}$ (in the sense of the statement's convention, so $\Delta^{f}_j(s,t)=\int_0^tf^{0}_j\,dW^{j}-\int_0^sf^{0}_j\,dW^{j}$) and similarly $\Delta^{h}_j(s,t)$. By bilinearity and claim 3 of Wiener Integrals Against a Vector Brownian Motion are Jointly Gaussian, for $a\le s\le t\le b$ and any $j,j'$,

using Additivity of the Riemann Integral on Adjacent Intervals, claim 4 of Componentwise Estimates, Transpose Identities, and Indefinite Riemann Integrals, and Extreme Value Theorem on a Compact Interval (the extended integrands agree with $f_j,h_j$ on $[s,t]\subseteq[a,b]$); degenerate intervals follow Mean-Square Riemann Integral of a Family of Random Variables.

Regularity. $(Y_t)$ is mean-square continuous: the time-integral part by claim 6 of Basic Properties of the Mean-Square Riemann Integral, the Wiener parts because $\lVert\Delta^{f}_j(s,t)\rVert_2^{2}=\operatorname{Var}(\Delta^{f}_j(s,t))\le(t-s)\max f_j^{2}$, and sums by the triangle inequality; likewise $(Z_t)$. $Y_a=y$ and $Z_a=z$ almost surely by the conventions. By Uniform Mean-Square Continuity on a Compact Interval, $\sup_t\lVert Y_t\rVert_2$ and $\sup_t\lVert Z_t\rVert_2$ are finite; fix $M_\alpha=\max_t\lVert\alpha_t\rVert_2$, $M_\beta=\max_t\lVert\beta_t\rVert_2$ (claim 4 of Basic Properties of the Mean-Square Riemann Integral and Extreme Value Theorem on a Compact Interval).

Claim 1. Put $\varphi(t)=\mathbb{E}[Y_tZ_t]$ and $\psi(r)=\mathbb{E}[\alpha_rZ_r]+\mathbb{E}[Y_r\beta_r]+\sum_jf_j(r)h_j(r)$.

Continuity. $|\varphi(t)-\varphi(s)|\le\lVert Y_t-Y_s\rVert_2\lVert Z_t\rVert_2+\lVert Y_s\rVert_2\lVert Z_t-Z_s\rVert_2$ (Cauchy-Schwarz), so $\varphi$ is continuous on $[a,b]$; similarly $|\mathbb{E}[\alpha_rZ_r]-\mathbb{E}[\alpha_{r'}Z_{r'}]|\le\lVert\alpha_r-\alpha_{r'}\rVert_2\lVert Z_r\rVert_2+\lVert\alpha_{r'}\rVert_2\lVert Z_r-Z_{r'}\rVert_2$, so $\psi$ is continuous.

Increment expansion. Fix $a\le s<t\le b$. Almost surely (claim 5 of Basic Properties of the Mean-Square Riemann Integral for the time parts), $Y_t-Y_s=\int_s^t\alpha_r\,dr+\sum_j\Delta^{f}_j(s,t)$ and similarly for $Z$. Then

First term: $\mathbb{E}[(\int_s^t\alpha)Z_s]=\int_s^t\mathbb{E}[\alpha_rZ_s]\,dr$ (claim 3 of Basic Properties of the Mean-Square Riemann Integral on $[s,t]$), and $\mathbb{E}[\Delta^{f}_j(s,t)Z_s]=\operatorname{Cov}(\Delta^{f}_j(s,t),Z_s)=0$ by the orthogonality hypothesis and centering. Second term symmetrically: $\mathbb{E}[Y_s(Z_t-Z_s)]=\int_s^t\mathbb{E}[Y_s\beta_r]\,dr$. Third term: expanding bilinearly, $\mathbb{E}[(\int_s^t\alpha)(\int_s^t\beta)]$ is bounded in absolute value by $(t-s)^{2}M_\alpha M_\beta$ (claims 4 and 6 of Basic Properties of the Mean-Square Riemann Integral and Cauchy-Schwarz); each mixed term $\mathbb{E}[(\int_s^t\alpha)\Delta^{h}_j]$ or $\mathbb{E}[\Delta^{f}_j(\int_s^t\beta)]$ is bounded by $(t-s)^{3/2}$ times a constant (Cauchy-Schwarz with the variance bound above); and $\sum_{j,j'}\mathbb{E}[\Delta^{f}_j\Delta^{h}_{j'}]=\sum_j\int_s^tf_jh_j\,dr$. Hence

for a constant $C$ independent of $s,t$.

Differentiability. Fix an interior point $s\in(a,b)$. For $t\downarrow s$, dividing by $t-s$: the error term tends to $0$, and

by monotonicity of the Riemann integral (via Agreement of the Riemann and Lebesgue Integrals for Continuous Functions on a Closed Interval and Linearity and Monotonicity of the Lebesgue Integral); the right side tends to $0$ as $t\downarrow s$ by continuity (Cauchy-Schwarz and mean-square continuity, as above). For the left-hand quotient, apply the increment expansion to the pair $(t',s)$ with $t'\uparrow s$: the frozen factors are then $Y_{t'},Z_{t'}$, and by Cauchy-Schwarz

uniformly over $r\in[t',s]$ as $t'\uparrow s$ (uniform mean-square continuity, Uniform Mean-Square Continuity on a Compact Interval, and the bounded norms $M_\alpha$, $\sup\lVert Z\rVert_2$), and similarly for $\mathbb{E}[Y_{t'}\beta_r]$; the same maximum estimate then gives the left-hand derivative. Hence $\varphi$ is differentiable at every interior point with $\varphi'=\psi$; $\varphi$ is continuous on $[a,b]$ and $\psi$ is continuous, so $\varphi$ is an antiderivative of $\psi$ on $[a,b]$, and Fundamental Theorem of Calculus, Part II in One Dimension, applied on $[a,t]$ for each $t$ (degenerate $t=a$ by convention), gives the asserted identity with $\varphi(a)=\mathbb{E}[Y_aZ_a]=\mathbb{E}[yz]$ (almost-sure equality preserves expectations of products).

Claim 2. The map $Q'\mapsto\operatorname{Cov}(\Delta,Q')$, $\Delta:=\int_a^tf\,dW^{j}-\int_a^sf\,dW^{j}$, is linear and continuous under mean-square limits: $|\operatorname{Cov}(\Delta,Q')-\operatorname{Cov}(\Delta,Q'')|\le\lVert\Delta\rVert_2\lVert Q'-Q''\rVert_2+|\mathbb{E}[\Delta]|\,|\mathbb{E}[Q'-Q'']|$, and both terms tend to $0$ along mean-square convergence (Cauchy-Schwarz; $|\mathbb{E}[V]|\le\lVert V\rVert_2$). Hence it suffices to check $\operatorname{Cov}(\Delta,Q')=0$ for $Q'$ ranging over the generators. For $Q'=1$: covariances with constants vanish. For $Q'=W^{j'}_r$ with $r\le s$: claim 3 of Wiener Integrals Against a Vector Brownian Motion are Jointly Gaussian and bilinearity give $\operatorname{Cov}(\Delta,W^{j'}_r)=\delta_{jj'}\bigl(\int_0^{\min(t,r)}f^{0}-\int_0^{\min(s,r)}f^{0}\bigr)=\delta_{jj'}\bigl(\int_0^{r}f^{0}-\int_0^{r}f^{0}\bigr)=0$. For $Q'=\xi^{i}$: choose the $\sigma(W^{j}_v:v\ge0)$-measurable versions of the two Wiener integrals (claim 1 of Wiener Integrals Against a Vector Brownian Motion are Jointly Gaussian; covariances are unchanged under almost-sure replacement), so $\Delta$ is measurable for a sub-$\sigma$-algebra of $\sigma(W^{j'}_v:\text{all }j',v)$, which is independent of $\sigma(\xi^{1},\dots,\xi^{l})$; hence $\Delta$ and $\xi^{i}$ are independent (closing remark of Sigma-Algebra Generated by Random Variables and Independence of Sigma-Algebras), and Expectation of a Product of Independent Random Variables gives $\mathbb{E}[\Delta\xi^{i}]=\mathbb{E}[\Delta]\mathbb{E}[\xi^{i}]$, so $\operatorname{Cov}(\Delta,\xi^{i})=0$. $\blacksquare$