Existence of Asymptotically Optimal-Value Observation-Driven Policies

Adopt the setting, hypotheses (H1)--(H4) and (C), and notation of the approximate Kalman filter and policy lemma, for the fluctuation LQG data of the stationary mean-field triple $(S,A,P)$, whose stationary co-state $P$ has value $P_0=(P^1_0,\dots,P^l_0)$ at time $0$: in particular the natural numbers $l\ge2$, $m\ge1$, $\tilde{l}\ge1$, the control set $\mathcal{A}$ (a nonempty subset of $\mathbb{R}^m$), the transition-rate family $\beta$, with control set $\mathcal{A}$, and the observation-rate family $\tilde{\beta}$, together with their extensions, the cost extension of the population cost data $(L,G)$, the horizon $T>0$, the matrices $\mathcal{E}_t$, $\mathcal{B}_t$, $\tilde{\mathcal{E}}_t$, $\Theta^\star_t$, $\tilde{\Theta}^\star_t$ of the fluctuation LQG data, the symmetrized coefficient matrices $Q_t$, $V_t$, $R_t$, $\hat{F}$ --- each $R_t$ symmetric positive definite with continuous inverse by conclusion 1 of the policy lemma --- the Riccati family $Z=(Z_t)_{t\in[0,T]}$ of (H2), fixed throughout, with $W_t=Z_t\mathcal{B}_t+\tfrac12V_t$, the matrix $\Pi_0$ of (H4), and the filter covariance $\Pi=(\Pi_t)_{t\in[0,T]}$ of conclusion 2 of the policy lemma. Throughout, a real-valued function on a subinterval $I$ of the real numbers $\mathbb{R}$ is called continuous on $I$ when it is continuous relative to $I$, both $I$ and the codomain $\mathbb{R}$ carrying the metric of the real line. For conclusion 2 below, which invokes the cost-limit proposition, and for the final sentence of conclusion 3, which combines conclusion 2 with the minimum identity, assume in addition the two hypotheses that the proposition carries beyond (H1)--(H4) and (C): that the control set $\mathcal{A}$ is convex, and hypothesis (H5) there, namely that there is a real $\varrho>0$ such that every $a\in\mathbb{R}^m$ with $|a-A_t|\le\varrho$ lies in $\mathcal{A}$, for every $t\in[0,T]$ (as that proposition records, these two together with (C) and the boundedness of the control-side open set $V\supseteq\mathcal{A}$ of the transition-rate extension --- the letter $V$ here being unrelated to the matrices $V_t$ and to the number $V^{*}$ defined below --- make $\mathcal{A}$ a closed, bounded, convex subset of $\mathbb{R}^m$). The definition of $V^{*}$, conclusion 1, and all of conclusion 3 except its final sentence do not use these two hypotheses. All entries of the maps $t\mapsto Z_t$ (part of (H2)), $t\mapsto\mathcal{B}_t$, $t\mapsto V_t$, $t\mapsto\Theta^\star_t$, and $t\mapsto R_t^{-1}$ (conclusion 1 of the policy lemma), and $t\mapsto\Pi_t$ (conclusion 2 there) are continuous on $[0,T]$; hence, with the matrix pairing $A'\cdot B'=\sum_{\gamma,\delta=1}^{l}A'^{\gamma\delta}B'^{\gamma\delta}$ of the cost-limit proposition, the map $t\mapsto Z_t\cdot\Theta^\star_t+(W_tR_t^{-1}W_t^{\top})\cdot\Pi_t$ is continuous on $[0,T]$ by sums and products of continuous functions. Define the real number

the integral being the Lebesgue integral over the compact interval $[0,T]$ of this continuous integrand, which agrees with its Riemann integral by claim 3 of that interval toolkit. Then:

1. (Existence of the policies.) For each natural number $N\ge1$ the approximate Kalman policy $h^N$ of conclusion 3 of the policy lemma is an observation-driven control policy with horizon $T$, control dimension $m$, and $\tilde{l}$ channels, and it is $\mathcal{A}$-valued; and for every $N$-agent driving system with $l$ states and $\tilde{l}$ observation channels there is a projected solution of the controlled $N$-agent dynamics on $[0,T]$ for $\beta$, $\tilde{\beta}$, that driving system, and $h^N$, as in conclusion 4(a) of the policy lemma --- the state processes, observation processes, and regular event of a solution for the modified family $\beta^{\sharp}$ and policy $h^{\sharp}$ of conclusion 3 there, with the control process replaced by the vector of its first $m$ components --- and every solution for $\beta$, $\tilde{\beta}$, that driving system, and $h^N$ is indistinguishable from it in the sense of the uniqueness theorem.

2. (Asymptotic value.) Suppose that for each $N\ge1$ such a driving system and projected solution are fixed, with empirical state measure $\Sigma^N_t$ as in the solution definition and state fluctuation process $\mathfrak{s}^N_t=\sqrt{N}(\Sigma^N_t-S_t)$, and that the initial-condition hypotheses (I1)--(I2) of the cost-limit proposition hold for these solutions: $\mathbb{E}[\mathfrak{s}^{N,\gamma}_0\mathfrak{s}^{N,\delta}_0]\to\Pi^{\gamma\delta}_0$ as $N\to\infty$ for all $\gamma,\delta\in\{1,\dots,l\}$, and $\sup\{\mathbb{E}[|\mathfrak{s}^N_0|^4]:N\ge1\}<\infty$, with the expectation. Let $J^N[h^N]$ be the $N$-agent cost of the solution at level $N$, let $J^{MF}$ be the mean-field cost of $(S,A)$, and set $\zeta_N=N(\mathbb{E}[\Sigma^N_0]-S_0)\in\mathbb{R}^l$ with the componentwise expectation, as in the cost-limit proposition. Then each $N(J^N[h^N]-J^{MF})+\sum_{\gamma=1}^{l}P^\gamma_0\zeta^\gamma_N$ is a well-defined real number and

3. (Optimality of the value.) For every linear-Gaussian state-observation model on $[0,T]$ with state dimension $l$, observation dimension $\tilde{l}$, and any Brownian dimension $m^\circ\ge1$, its data written $(A^\circ,\varepsilon^\circ,\tilde{E}^\circ,\tilde{\varepsilon}^\circ,\xi,W^\circ)$ as in the identification corollary and matched to the fluctuation LQG data as in hypotheses (i)--(iii) there --- a class of coefficient data that is nonempty, by conclusion 3 of that corollary, already for Brownian dimension $l\cdot l+\tilde{l}$ --- with the control dimension of the controlled system taken to be $m$, the control matrix assignment sending each $t\in[0,T]$ to $\mathcal{B}_t$, and the cost data $Q_t$, $\tfrac12V_t$ (entrywise), $R_t$, $\hat{F}$, all as in the corollary, the linear-quadratic-Gaussian cost $J$ attains a minimum over the extended admissible controls with values in $\mathbb{R}^{m}$, and

In particular, under the hypotheses of conclusion 2 --- the driving systems and projected solutions fixed there, (I1)--(I2), and the two additional hypotheses --- the recentred limiting cost of conclusion 2 along the approximate Kalman policies equals this minimal LQG cost.