The Limiting Cost Along the Approximate Kalman Policy Is the Optimal Value of the Fluctuation LQG Problem

Adopt the setting, hypotheses (H1)--(H4) and (C), and notation of the approximate Kalman filter and policy lemma for the fluctuation LQG data of the stationary mean-field triple $(S,A,P)$, whose stationary co-state $P$ has components $P^\gamma_t$ ($\gamma\in\{1,\dots,l\}$, $t\in[0,T]$): in particular the natural numbers $l\ge2$, $m\ge1$, $\tilde{l}\ge1$, the horizon $T>0$, the matrices $\mathcal{E}_t$, $\mathcal{B}_t$, $\tilde{\mathcal{E}}_t$, $\Theta^\star_t$, $\tilde{\Theta}^\star_t$ of the fluctuation LQG data, the symmetrized coefficient matrices $Q_t$, $V_t$, $R_t$, $\hat{F}$, the Riccati family $Z=(Z_t)_{t\in[0,T]}$ of (H2) with $W_t=Z_t\mathcal{B}_t+\tfrac12 V_t$, the matrix $\Pi_0$ of (H4), and the filter covariance $\Pi=(\Pi_t)_{t\in[0,T]}$ of conclusion 2 of the policy lemma. For real matrices $A'$ and $B'$ with $l$ rows and $l$ columns write $A'\cdot B'=\sum_{\gamma,\delta=1}^{l}A'^{\gamma\delta}B'^{\gamma\delta}$, as in the cost-limit proposition.

Throughout, a real-valued function on a subinterval $I$ of the real numbers $\mathbb{R}$ is called continuous on $I$ when it is continuous relative to $I$, both $I$ and the codomain $\mathbb{R}$ carrying the metric of the real line. For the final assertion of conclusion 2 below, which invokes the cost-limit proposition, assume in addition the two hypotheses that proposition carries beyond (H1)--(H4) and (C): that the control set $\mathcal{A}$ is convex, and hypothesis (H5) there, namely that there is a real $\varrho>0$ such that every $a\in\mathbb{R}^m$ with $|a-A_t|\le\varrho$ lies in $\mathcal{A}$, for every $t\in[0,T]$. Conclusions 1 and 3 do not use these two.

Consider moreover a linear-Gaussian state-observation model on $[0,T]$ whose state dimension and observation dimension (the numbers $l$ and $\tilde{l}$ of the model definition) are the present $l$ and $\tilde{l}$, and whose Brownian dimension (the number $m$ of the model definition) is a natural number $m^\circ\ge1$, the letter $m$ remaining the control dimension of the fluctuation setting; write $(A^\circ,\varepsilon^\circ,\tilde{E}^\circ,\tilde{\varepsilon}^\circ,\xi,W^\circ)$ for its data, written $(A,\varepsilon,\tilde{E},\tilde{\varepsilon},\xi,W)$ in the model definition, the letters $A$ and $W$ being otherwise engaged here and the letters $\varepsilon$ and $\tilde{\varepsilon}$ being freed for the notation of the cost-limit proposition, whose filter error is written $\varepsilon^N$. Assume, with the matrix product and transpose:

(i) $A^\circ(t)=\mathcal{E}_t$ and $\tilde{E}^\circ(t)=\tilde{\mathcal{E}}_t$ for every $t\in[0,T]$;

(ii) $\varepsilon^\circ(t)\,\varepsilon^\circ(t)^{\top}=\Theta^\star_t$ and $\tilde{\varepsilon}^\circ(t)\,\tilde{\varepsilon}^\circ(t)^{\top}=\tilde{\Theta}^\star_t$ for every $t\in[0,T]$ --- so the matrices $\Theta(t)=\varepsilon^\circ(t)\varepsilon^\circ(t)^{\top}$ and $\tilde{\Theta}(t)=\tilde{\varepsilon}^\circ(t)\tilde{\varepsilon}^\circ(t)^{\top}$ of the model definition equal $\Theta^\star_t$ and $\tilde{\Theta}^\star_t$;

(iii) $\mathbb{E}[\xi^{\gamma}]=0$ and $\operatorname{Cov}(\xi^{\gamma},\xi^{\delta})=\Pi_0^{\gamma\delta}$ for all $\gamma,\delta\in\{1,\dots,l\}$, with the expectation and the covariance.

Take the control dimension $k$ of the controlled system to be $m$, and the control matrix assignment to be $B(t)=\mathcal{B}_t$ ($0\le t\le T$) --- an assignment of real matrices with $l$ rows and $m$ columns whose entries are continuous by conclusion 1 of the policy lemma. Take the cost data to be

$$Q^\circ(t)=Q_t,\qquad V^\circ(t)=\tfrac12\,V_t\ \text{(entrywise)},\qquad R^\circ(t)=R_t,\qquad F^\circ=\hat{F}\qquad(0\le t\le T)$$

--- these are cost data with every $R^\circ(t)$ positive definite, by conclusion 1 of the policy lemma and the entry formulas recorded there. Write $J[\alpha]$ for the linear-quadratic-Gaussian cost of each extended admissible control $\alpha$ with values in $\mathbb{R}^{m}$ for these data. Then:

1. (Identification of the LQG ingredients.) The family $Z$ is a symmetric continuous solution of the backward Riccati equation of the completion-of-squares theorem for these data, with $Z_t\,B(t)+V^\circ(t)=W_t$ for every $t\in[0,T]$; and the initial covariance matrix $P^{\mathrm{KB}}_0$ (written $P_0$ in that theorem) and the covariance assignment of claim 1 of the Kalman--Bucy filter theorem, formed for this model, satisfy: $P^{\mathrm{KB}}_0=\Pi_0$, and the covariance assignment assigns to each $t\in[0,T]$ exactly the matrix $\Pi_t$ of the policy lemma.

2. (The minimal cost is the limiting cost.) The optimal value $V^{*}$ of the separation theorem, formed for these data and the solution $Z$ of conclusion 1, is the minimum of $J$ over all extended admissible controls with values in $\mathbb{R}^{m}$, by the separation theorem over extended admissible controls; it is attained by the closed-loop feedback control of the closed-loop feedback lemma; and

$$\min J\;=\;V^{*}\;=\;Z_0\cdot\Pi_0+\int_{[0,T]}\Big(Z_t\cdot\Theta^\star_t+\big(W_t R_t^{-1}W_t^{\top}\big)\cdot\Pi_t\Big)\,dt\,,$$

the integral being the Lebesgue integral over the compact interval $[0,T]$ of an integrand that is continuous on $[0,T]$ --- all entries of $t\mapsto Z_t$ are continuous (part of (H2)), all entries of $t\mapsto\Theta^\star_t$, $t\mapsto R_t^{-1}$, and $t\mapsto\Pi_t$ are continuous by conclusions 1 and 2 of the policy lemma, and the entries of $t\mapsto W_t=Z_t\mathcal{B}_t+\tfrac12V_t$ and the integrand itself are built from these and the continuous entries of $t\mapsto\mathcal{B}_t$ and $t\mapsto V_t$ (conclusion 1 again) by sums and products of continuous functions --- and whose Lebesgue and Riemann integrals agree by claim 3 of that toolkit; the right-hand side above is precisely the right-hand side of the limit identity of the cost-limit proposition. In particular, if moreover for each natural number $N\ge1$ a driving system and a projected solution along the approximate Kalman policy $h^N$ are fixed as in the cost-limit proposition, with $J^N[h^N]$, $J^{MF}$, and $\zeta_N$ as there, and the initial-condition hypotheses (I1)--(I2) there hold, then

$$\lim_{N\to\infty}\Big(N\big(J^N[h^N]-J^{MF}\big)+\sum_{\gamma=1}^{l}P^{\gamma}_0\,\zeta^{\gamma}_N\Big)\;=\;\min J\,.$$

3. (Realizability of the coefficients.) For the Brownian dimension $m^\circ=l\cdot l+\tilde{l}$ there exist assignments $\varepsilon^\circ$ and $\tilde{\varepsilon}^\circ$ of real matrices with $l$ rows and $m^\circ$ columns, respectively $\tilde{l}$ rows and $m^\circ$ columns, to the points of $[0,T]$, all of whose entries are continuous on $[0,T]$, such that for every $t\in[0,T]$

$$\varepsilon^\circ(t)\,\varepsilon^\circ(t)^{\top}=\Theta^\star_t,\qquad \tilde{\varepsilon}^\circ(t)\,\tilde{\varepsilon}^\circ(t)^{\top}=\tilde{\Theta}^\star_t,\qquad \varepsilon^\circ(t)\,\tilde{\varepsilon}^\circ(t)^{\top}=0,$$

the last matrix being the zero matrix with $l$ rows and $\tilde{l}$ columns. Such assignments satisfy hypothesis (ii) and conditions (i) and (ii) of the model definition (every $\tilde{\Theta}^\star_t$ being symmetric positive definite by conclusion 1 of the policy lemma), so hypotheses (i)--(iii) concern a nonempty class of coefficient data; the model's remaining stochastic data --- the Brownian motion and the jointly Gaussian initial vector $\xi$ --- stay hypothesized exactly as in the model definition, as everywhere in this chain.