Proof of Existence of Asymptotically Optimal-Value Observation-Driven Policies

Well-definedness of $V^{*}$. All entries of $t\mapsto Z_t$ are continuous on $[0,T]$ as part of hypothesis (H2); all entries of $t\mapsto\mathcal{B}_t$, $t\mapsto V_t$, $t\mapsto\Theta^\star_t$, and $t\mapsto R_t^{-1}$ are continuous by conclusion 1 of the approximate Kalman filter and policy lemma, and all entries of $t\mapsto\Pi_t$ are continuous by conclusion 2 there. The entries of $t\mapsto W_t=Z_t\mathcal{B}_t+\tfrac12V_t$ and the map $t\mapsto Z_t\cdot\Theta^\star_t+(W_tR_t^{-1}W_t^{\top})\cdot\Pi_t$ are built from these by finitely many sums and products of continuous functions, hence are continuous on $[0,T]$; the Lebesgue integral over the compact interval $[0,T]$ of the latter map therefore exists and agrees with its Riemann integral, so $V^{*}$ is a well-defined real number.

Conclusion 1. By conclusion 3 of the approximate Kalman filter and policy lemma, $h^N$ is an observation-driven control policy with horizon $T$, control dimension $m$, and $\tilde{l}$ channels, the family $\beta^{\sharp}$ defined there is a transition-rate family on $l$ states with control dimension $m+l$ and rate bound $B$, and the family $h^{\sharp}$ defined there is an observation-driven control policy with horizon $T$, control dimension $m+l$, and $\tilde{l}$ channels. Given an $N$-agent driving system with $l$ states and $\tilde{l}$ observation channels, solutions of the controlled $N$-agent dynamics on $[0,T]$ for $\beta^{\sharp}$, $\tilde{\beta}$, that driving system, and $h^{\sharp}$ exist by the existence and uniqueness theorem for the controlled $N$-agent dynamics, as recorded in conclusion 4 of the policy lemma. By conclusion 4(a) of the policy lemma, the projection of such a solution --- the same state and observation processes and regular event, with the control process replaced by the vector of its first $m$ components --- is a solution of the controlled $N$-agent dynamics on $[0,T]$ for $\beta$, $\tilde{\beta}$, the same driving system, and $h^N$, and every solution for these data is indistinguishable from it in the sense of the uniqueness theorem. This proves conclusion 1.

Conclusion 2. Fix, for each $N\ge1$, a driving system and a projected solution as in conclusion 1, with empirical state measure $\Sigma^N_t$ and state fluctuation process $\mathfrak{s}^N_t$, and assume (I1)--(I2). The hypotheses of the cost-limit proposition then hold: its setting, hypotheses (H1)--(H4), and notation are those adopted here, its per-level data --- the driving system, the projected solution as in conclusion 4 of the policy lemma, the $N$-agent cost $J^N[h^N]$, the mean-field cost $J^{MF}$, and the vector $\zeta_N$ --- are those fixed in the statement, and its initial-condition hypotheses (I1)--(I2), spelled out in the statement of conclusion 2, are assumed. The proposition asserts that each $N(J^N[h^N]-J^{MF})+\sum_{\gamma=1}^{l}P^\gamma_0\zeta^\gamma_N$ is a well-defined real number and that

and the right-hand side is $V^{*}$ by definition. This proves conclusion 2.

Conclusion 3. Let a linear-Gaussian state-observation model on $[0,T]$ with state dimension $l$, observation dimension $\tilde{l}$, and Brownian dimension $m^\circ\ge1$, matched to the fluctuation LQG data as in hypotheses (i)--(iii) of the identification corollary, be given; take the control dimension of the controlled system to be $m$, the control matrix assignment sending each $t\in[0,T]$ to $\mathcal{B}_t$, and the cost data $Q_t$, $\tfrac12V_t$ (entrywise), $R_t$, $\hat{F}$ --- exactly the choices made in the corollary. By conclusion 2 of the corollary, the minimum of the linear-quadratic-Gaussian cost $J$ over the extended admissible controls with values in $\mathbb{R}^{m}$ exists (it is attained by the closed-loop feedback control of the closed-loop feedback lemma) and

By conclusion 3 of the corollary, the class of matched coefficient data is nonempty. The final sentence of conclusion 3 follows by combining the displayed identity with the limit of conclusion 2. This proves conclusion 3. $\square$