Proof of Completion of Squares for the Linear-Quadratic-Gaussian Cost

Fix an admissible control $\alpha$ and write $Y:=X^{\alpha}$ for its controlled state and $\beta_r:=A(r)Y_r+B(r)\alpha_r$ (componentwise, matrix-vector products) for the drift; each component family of $\beta$ is mean-square continuous by claims 1-2 of Basic Properties of the Mean-Square Riemann Integral. Throughout we use the componentwise algebra of dot products, matrix-vector products, and transposes from claim 3 of Componentwise Estimates, Transpose Identities, and Indefinite Riemann Integrals, and the identity $(UV)z=U(Vz)$ for matrices $U,V$ and vectors $z$, which follows from the defining index formulas of the matrix product and matrix-vector product by exchanging finite sums.

Step 1: evolution of the second moments. For $1\le i,j\le l$ set $M_{ij}(t):=\mathbb{E}[Y^{i}_tY^{j}_t]$. Since $Y$ is a mean-square solution, $Y^{i}_t$ is almost surely equal, at each $t$, to $\xi^{i}+\int_0^t\beta^{i}_r\,dr+\sum_{j'}\int_0^t\varepsilon_{ij'}(r)\,dW^{j'}_r$, i.e., to a process of the integral form of Second-Moment Evolution for Processes of Integral Form with $a=0$, $b=T$ (so that its hypothesis $0\le a<b$ holds). We instantiate that lemma with its two initial values taken to be $\xi^{i}$ and $\xi^{j}$, its two drift families taken to be $(\beta^{i}_r)_r$ and $(\beta^{j}_r)_r$, its two processes taken to be versions of $Y^{i}$ and $Y^{j}$, and its deterministic Wiener integrands taken to be $\varepsilon_{ij'}$ and $\varepsilon_{jj'}$ ($1\le j'\le m$); the symbols $\alpha$, $Y$, $Z$ appearing in that lemma's statement are placeholders unrelated to the control $\alpha$ and the Riccati solution $Z$ of the present theorem.

We verify the orthogonality hypothesis of claim 1 of Second-Moment Evolution for Processes of Integral Form for each pair $(i,j)$: for $0\le s<t\le T$ and each $j'$, the Wiener-integral increment $\int_0^t\varepsilon_{ij'}\,dW^{j'}-\int_0^s\varepsilon_{ij'}\,dW^{j'}$ has vanishing covariance with $Y^{j}_s$ by claim 3 of Brownian Increments After a Time are Independent of the Model Past, because $Y^{j}_s$ is almost surely equal to an $\mathcal{H}_s$-measurable square-integrable random variable: indeed $Y^{j}_s=X^{j}_s+c^{j}_s$ almost surely by claim 2 of Superposition Decomposition of the Controlled State and Observations; $X^{j}_s$ is a mean-square limit of finite linear combinations of the $\xi^{i'}$ and the $W^{j''}_r$ ($r\le s$) by claim 1 of Gaussian and Span Structure of the Linear-Gaussian State-Observation Model, all of which are $\mathcal{H}_s$-measurable, so $X^{j}_s$ is almost surely equal to an $\mathcal{H}_s$-measurable square-integrable random variable by claim 2 of The Closed Mean-Square Span of a Family of Random Variables; and $c^{j}_s$ is almost surely equal to a $\mathcal{G}_s$-measurable one by claim 3 of Superposition Decomposition of the Controlled State and Observations, with $\mathcal{G}_s\subseteq\mathcal{H}_s$ as noted in Brownian Increments After a Time are Independent of the Model Past. Both the orthogonality hypothesis and the conclusion of claim 1 of Second-Moment Evolution for Processes of Integral Form involve the processes only through covariances and expectations at fixed times, which are unchanged when a random variable is replaced by an almost surely equal one; the lemma therefore applies to our versions and gives: each $M_{ij}$ is continuous and

with $\mu_{ij}$ continuous, since $\sum_{j'}\varepsilon_{ij'}(r)\varepsilon_{jj'}(r)=\bigl(\varepsilon(r)\varepsilon(r)^{\top}\bigr)_{ij}=\Theta_{ij}(r)$. Also $M_{ij}(0)=\mathbb{E}[\xi^{i}\xi^{j}]$, since $Y_0=\xi$ almost surely (the integrals over the degenerate interval vanish).

Step 2: the backward equation in indefinite form. Write $\zeta(r):=A(r)^{\top}Z(r)+Z(r)A(r)-(Z(r)B(r)+V(r))R(r)^{-1}(Z(r)B(r)+V(r))^{\top}+Q(r)$; its entries are continuous (Sums and Products of Continuous Real-Valued Functions, with the continuity of $R^{-1}$ from the statement). By claim 4 of Componentwise Estimates, Transpose Identities, and Indefinite Riemann Integrals, $\int_t^T\zeta_{ij}=\int_0^T\zeta_{ij}-\int_0^t\zeta_{ij}$, so the backward Riccati equation reads entrywise

and evaluating the original equation at $t=T$ gives $Z(T)=F$ by the degenerate-interval convention.

Step 3: product rule. Set $\varphi(t):=\sum_{i,j}Z_{ij}(t)M_{ij}(t)$. Applying claim 1 of Product Rule and Reflection for Indefinite Riemann Integrals to each pair (with the two functions there equal to $Z_{ij}$ and $M_{ij}$) and summing (linearity of the Riemann integral for continuous integrands, via Agreement of the Riemann and Lebesgue Integrals for Continuous Functions on a Closed Interval and Linearity and Monotonicity of the Lebesgue Integral),

Step 4: algebraic identification of the integrand. Fix $r$ and drop it from the notation. By the index formulas of the dot product and matrix-vector product and linearity of the expectation, $\sum_{i,j}\zeta_{ij}M_{ij}=\mathbb{E}[Y\cdot(\zeta Y)]$, $\sum_{i,j}Z_{ij}\,\mathbb{E}[\beta^{i}Y^{j}]=\mathbb{E}[\beta\cdot(ZY)]$, and $\sum_{i,j}Z_{ij}\,\mathbb{E}[Y^{i}\beta^{j}]=\mathbb{E}[Y\cdot(Z\beta)]$, as in claim 1 of Expected Bilinear Forms: Trace Formula and Mean-Square Continuity. Since $Z$ is symmetric, $\beta\cdot(ZY)=(Z^{\top}\beta)\cdot Y=(Z\beta)\cdot Y=Y\cdot(Z\beta)$, so

using claim 4 of Basic Properties of the Trace and the symmetry of $Z$ for $\sum_{i,j}Z_{ij}\Theta_{ij}=\operatorname{tr}(Z^{\top}\Theta)=\operatorname{tr}(Z\Theta)$. Substituting $\beta=AY+B\alpha$ (almost surely) and using $Z(AY)=(ZA)Y$, $Z(B\alpha)=(ZB)\alpha$:

Also, pointwise, $Y\cdot((A^{\top}Z)Y)=\bigl((A^{\top}Z)^{\top}Y\bigr)\cdot Y=\bigl((Z A) Y\bigr)\cdot Y$ (using $(A^{\top}Z)^{\top}=Z^{\top}A=ZA$), so $\mathbb{E}[Y\cdot((A^{\top}Z)Y)]=\mathbb{E}[Y\cdot((ZA)Y)]$. Hence, expanding $\zeta$,

where $\Xi:=(ZB+V)R^{-1}(ZB+V)^{\top}$ (the two terms $\mathbb{E}[Y\cdot((ZA)Y)]$ from $\mu$ cancel against $-\mathbb{E}[Y\cdot((A^{\top}Z)Y)]-\mathbb{E}[Y\cdot((ZA)Y)]$ from $-\zeta$).

Step 5: assembling the cost. By The Linear-Quadratic-Gaussian Cost Functional and Step 2 ($Z(T)=F$, so $\mathbb{E}[Y_T\cdot(FY_T)]=\sum_{i,j}F_{ij}M_{ij}(T)=\varphi(T)$ by claim 1 of Expected Bilinear Forms: Trace Formula and Mean-Square Continuity),

Inserting the expression for $\varphi(T)$ from Steps 3-4 and cancelling $\mathbb{E}[Y\cdot(QY)]$, and combining $2\mathbb{E}[Y\cdot(V\alpha)]+2\mathbb{E}[Y\cdot((ZB)\alpha)]=2\mathbb{E}[Y\cdot((ZB+V)\alpha)]$ (componentwise distributivity),

all integrand splittings being justified since each summand is continuous in $t$ (claim 2 of Expected Bilinear Forms: Trace Formula and Mean-Square Continuity) and the Riemann integral of continuous functions is linear (Agreement of the Riemann and Lebesgue Integrals for Continuous Functions on a Closed Interval, Linearity and Monotonicity of the Lebesgue Integral).

Step 6: completing the square. Fix $t$ and write $q:=\alpha_t-\Gamma Y_t$. Pointwise on $\Omega$, using $R\Gamma=-(ZB+V)^{\top}$ (from $RR^{-1}=I$ and associativity), $\Gamma^{\top}=-(ZB+V)R^{-1}$ (transpose rules and symmetry of $R^{-1}$, from Invertibility of Symmetric Positive Definite Matrices), and $\Gamma^{\top}R\Gamma=(ZB+V)R^{-1}RR^{-1}(ZB+V)^{\top}=\Xi$:

where we used $-\alpha_t\cdot(R\Gamma Y_t)=\alpha_t\cdot\bigl((ZB+V)^{\top}Y_t\bigr)=\bigl((ZB+V)\alpha_t\bigr)\cdot Y_t$ and the symmetric computation for the third term, and $(\Gamma Y_t)\cdot(R\Gamma Y_t)=Y_t\cdot\bigl(\Gamma^{\top}R\Gamma Y_t\bigr)$, all by claim 3 of Componentwise Estimates, Transpose Identities, and Indefinite Riemann Integrals. Taking expectations, the last integrand of Step 5 equals $\mathbb{E}[q\cdot(Rq)]$ at each $t$; this function of $t$ is continuous by claim 2 of Expected Bilinear Forms: Trace Formula and Mean-Square Continuity, the components of $(\alpha_t-\Gamma(t)Y_t)_t$ being mean-square continuous (claims 1-2 of Basic Properties of the Mean-Square Riemann Integral).

Step 7: the initial term. By claim 1 of Expected Bilinear Forms: Trace Formula and Mean-Square Continuity applied to $\xi$ and the symmetric matrix $Z(0)$, and since $M_{ij}(0)=\mathbb{E}[\xi^{i}\xi^{j}]$ (Step 1),

$P_0$ being exactly the covariance matrix of $\xi$ (claim 1 of The Kalman-Bucy Filter Equation and Its Solution). Combining Steps 5-7 yields the asserted formula. $\square$