The Fluctuation LQG Value Is the Asymptotically Optimal Value of the Recentred N-Agent Cost over Admissible Families of Observation-Driven Policies, and Is Attained by the Approximate Kalman Policies, under Deterministic Initial States

Common data. Adopt the common data and the standing hypotheses of Asymptotic Lower Bound for the Recentred N-Agent Cost without Uniform Control-Moment Hypotheses, and the notation of Injection Certificates on the Trimmed Synthetic Copy for the Family of N-Agent Solutions: the Law-Transported Van Trees Certificate Hypothesis Holds under Deterministic Initial States, Uniform Observation Positivity and a C^2 Observation Extension for them: the natural numbers $l\ge2$, $m\ge1$, $\tilde{l}\ge1$; the affine-controlled transition-rate family $(\beta_{0},\beta_{1})$ on $l$ states with nonempty convex control set $\mathcal{A}\subseteq\mathbb{R}^{m}$, compact for the topology $\mathcal{T}_{d_{E}}$ of the Euclidean distance $d_{E}$ (the collection of subsets of $\mathbb{R}^{m}$ that are open in $(\mathbb{R}^{m},d_{E})$, a topology by The Open Subsets of a Metric Space Form a Topology; this is the topology with respect to which compactness of the control set is understood in Affine-Controlled Transition-Rate Family and throughout the adopted setting), with control bound $R=\sup_{a\in\mathcal{A}}|a|$, and its transition-rate family $\beta$ with rate bound $B$; the twice continuously differentiable extension $(U,V,\bar\beta)$ of $\beta$ with derivative bound $K$ and its extended aggregate state drift $\bar{b}$; the observation-rate family $\tilde\beta$ with $\tilde{l}$ channels and rate bound $\tilde{B}$ and its aggregate observation drift $\tilde{b}$; the horizon $T>0$; the population cost data $(L,G)$, convex in the control on $\mathcal{A}$, with their twice continuously differentiable extension, written $(U_{c},\bar{L},\bar{G})$ here (its open set is written $W$ in Fluctuation LQG Data of a Stationary Mean-Field Triple and $U_{c}$ in The Approximate Kalman Filter and Policy for the Controlled N-Agent Dynamics; the letter $W$ denotes the observation record below); the stationary mean-field triple $(S,A,P)$ for these data with initial point $S_{0}$ in the probability simplex $\Delta^{l}$, whose co-state has the components $P^{1}_{0},\dots,P^{l}_{0}$ at time $0$; the aggregate fluctuation covariance $\Theta$ of $\beta$ with $\Theta^{\star}_{t}=\Theta(S_{t},A_{t})$; the matrices $E_{t}$, $\mathsf{B}_{t}$, $Q_{t}$, $V_{t}$, $R_{t}$ and $\hat{F}$ of the completion-of-squares theorem ($E_{t}$ and $\mathsf{B}_{t}$ formed from the partial derivatives of $\bar{b}$ along $(S,A)$, and $Q_{t}$, $V_{t}$, $R_{t}$, $\hat{F}$ from the fluctuation Hessian coefficients of $(U,V,\bar\beta)$, $(U_{c},\bar{L},\bar{G})$ and $(S,A,P)$), its entry pairing $x\cdot My$, and its hypothesis (H2) with the Riccati family $Z=(Z_{t})_{t\in[0,T]}$, part of the common data; and the standing hypotheses of Asymptotic Lower Bound for the Recentred N-Agent Cost without Uniform Control-Moment Hypotheses — hypotheses (A) and (U) of Quadratic Growth of the Mean-Field Hamiltonian in the Control along a Stationary Mean-Field Triple, (JC) of Localized Joint Coercivity of the Recentred N-Agent Cost Integrand under a Positive-Definite Fluctuation Hessian with constant $c_{J}>0$, (H2), (LipC) of Pointwise-in-Time Tracking of the Mean-Field Flow and Cost along the Realized Control of the Controlled N-Agent Dynamics, the optimality $[A]\in\mathcal{M}^{*}_{S_{0}}$ and (TG) of Quadratic Expansion Bounds for the Mean-Field To-Go Value along a Stationary Mean-Field Triple, and the standing hypothesis on $S^{*}=S$ of The Block Cascade of Anchored Good-Set Clocks: Adapted Good Sets, Matched Escape Bounds, and the Energy Ledger — each of which, on inspection of the cited items, is a condition on the common data and the triple only, so that it is satisfied by every family of solutions for these data (the proof of claim 2 uses only this). Assume in addition:

(OC) $\underline{b}>0$ is a real number with $\tilde{b}^{\upsilon}(x)\ge\underline{b}$ for all $x\in\Delta^{l}$ and all $\upsilon\in\{1,\dots,\tilde{l}\}$ (hypothesis (OC) of Injection Certificates on the Trimmed Synthetic Copy for the Family of N-Agent Solutions: the Law-Transported Van Trees Certificate Hypothesis Holds under Deterministic Initial States, Uniform Observation Positivity and a C^2 Observation Extension);

(X$'$) $(\tilde{U},\bar{\tilde\beta})$ is a twice continuously differentiable extension of $\tilde\beta$ with derivative bound $\tilde{K}\ge0$;

(H5) there is a real $\varrho_{\mathcal{A}}>0$ such that every $a\in\mathbb{R}^{m}$ with $|a-A_{t}|\le\varrho_{\mathcal{A}}$ lies in $\mathcal{A}$, for every $t\in[0,T]$ (hypothesis (H5) of Mean-Square Error Covariance of the Approximate Kalman Filter, assumed in Cost Limit Along the Approximate Kalman Policy; the subscript distinguishes $\varrho_{\mathcal{A}}$ from the near-field radius $\varrho$ and the certificate measure $\varrho_{0}$ of the cited settings).

Hypothesis (H1) of the completion-of-squares theorem — there is a real $r>0$ with $a\cdot R_{t}a\ge r|a|^{2}$ for every $t\in[0,T]$ and $a\in\mathbb{R}^{m}$ — holds with $r=c_{J}$ under (JC), as Ledger Decomposition of the Recentred N-Agent Cost over the Block Cascade and Its Near-Field Filtering Lower Bound records.

The fluctuation LQG data and the value. Let $\mathcal{E}_{t}$, $\mathcal{B}_{t}$, $\tilde{\mathcal{E}}_{t}$, $\Theta^{\star}_{t}$ and $\tilde{\Theta}^{\star}_{t}$ ($t\in[0,T]$) be the state matrix, control matrix, observation matrix, state noise covariance and observation noise covariance of the fluctuation LQG data of $(S,A,P)$ relative to the extensions $(U,V,\bar\beta)$, $(U_{c},\bar{L},\bar{G})$ and $(\tilde{U},\bar{\tilde\beta})$ (the state noise covariance is the matrix $\Theta^{\star}_{t}$ already named, the state matrix $\mathcal{E}_{t}$ coincides with $E_{t}$ and the control matrix $\mathcal{B}_{t}$ with $\mathsf{B}_{t}$, by the identical defining formulas), and put, for $t\in[0,T]$,

$$W_{t}=Z_{t}\mathsf{B}_{t}+\tfrac12V_{t},\qquad \Xi_{t}=W_{t}R_{t}^{-1}W_{t}^{\top},\qquad \tilde{D}_{t}=\tilde{\mathcal{E}}_{t}^{\top}(\tilde{\Theta}^{\star}_{t})^{-1}\tilde{\mathcal{E}}_{t},$$

with the matrix product, the transpose and the matrix inverse ($R_{t}$ is invertible under (H1), and $\tilde{\Theta}^{\star}_{t}$ under (OC); claim 1 below records both). Let $\Pi=(\Pi_{t})_{t\in[0,T]}$ be the solution of the Kalman covariance Riccati equation on $[0,T]$ with the data $A=\mathcal{E}$, $C=\Theta^{\star}$, $D=\tilde{D}$ and the zero matrix as initial value (the datum written $P_{0}$ in that theorem; claim 1 below records that the theorem applies), and define the real number

$$V^{*}=\int_{[0,T]}\Bigl(\sum_{\gamma=1}^{l}\sum_{\delta=1}^{l}Z^{\gamma\delta}_{t}\,\Theta^{\star\gamma\delta}_{t}+\sum_{\gamma=1}^{l}\sum_{\delta=1}^{l}\Xi^{\gamma\delta}_{t}\,\Pi^{\gamma\delta}_{t}\Bigr)\,dt ,$$

the Lebesgue integral over the compact interval $[0,T]$ of an integrand that claim 1 records to be continuous. Write $|\cdot|$ for the Euclidean norm and $\lim_{N\to\infty}$ for the limit of a real sequence.

Families of solutions; admissibility. A family of solutions for the common data consists of, for every natural number $N\ge1$: an $N$-agent driving system $(\Omega^{\mathrm{ag}},\mathcal{F}^{\mathrm{ag}},P^{\mathrm{ag}})$ with $l$ states and $\tilde{l}$ observation channels, with expectation $\mathbb{E}$; an $\mathcal{A}$-valued observation-driven control policy $h^{(N)}$ with horizon $T$, control dimension $m$ and $\tilde{l}$ channels; and a solution of the controlled $N$-agent dynamics on $[0,T]$ for $\beta$, $\tilde\beta$, that driving system and $h^{(N)}$, with regular event $\Omega_{0}$, empirical state measure $\Sigma$, observation record $W$, observation filtration $(\mathcal{G}_{t})_{t\in[0,T]}$, state fluctuation $\mathfrak{s}_{t}=\sqrt{N}(\Sigma_{t}-S_{t})$, $\kappa_{0}=1+\mathbb{E}[|\mathfrak{s}_{0}|^{4}]$, and recentred cost

$$\mathcal{J}_{N}=N\bigl(J^{N}[h^{(N)}]-J^{MF}\bigr)+\sum_{\gamma=1}^{l}P^{\gamma}_{0}\,\zeta^{\gamma}_{N},\qquad \zeta_{N}=N\bigl(\mathbb{E}[\Sigma_{0}]-S_{0}\bigr),$$

of First-Order Expansion of the Recentred N-Agent Cost about a Stationary Mean-Field Triple and Its Coercive Lower Bound, formed with the $N$-agent cost $J^{N}[h^{(N)}]$ of the $N$-th solution (finite by conclusion (b) of that lemma under (A)) and the mean-field cost $J^{MF}$ of $(S,A)$; the dependence of these objects on $N$ is suppressed. Such a family is called admissible if it satisfies the hypotheses that Asymptotic Lower Bound for the Recentred N-Agent Cost without Uniform Control-Moment Hypotheses and Injection Certificates on the Trimmed Synthetic Copy for the Family of N-Agent Solutions: the Law-Transported Van Trees Certificate Hypothesis Holds under Deterministic Initial States, Uniform Observation Positivity and a C^2 Observation Extension impose on a family beyond the common data and beyond hypothesis (I) of the former, which claim 2 below derives from (D0):

(I$'$) there is a real $\kappa^{\sharp}\ge1$ with $\kappa_{0}\le\kappa^{\sharp}$ for every $N\ge1$;

(CB) there is a real $\mathcal{J}^{\sharp}\ge0$ with $\mathcal{J}_{N}\le\mathcal{J}^{\sharp}$ for every $N\ge1$;

(D0) there are points $\mathsf{x}^{N}_{0}$ in the aggregate lattice $\mathbb{G}_{N}\subseteq\Delta^{l}$ with $P^{\mathrm{ag}}(\Sigma_{0}=\mathsf{x}^{N}_{0})=1$ for every $N\ge1$, such that the real sequence $(\sqrt{N}\,|\mathsf{x}^{N}_{0}-S_{0}|)_{N\ge1}$ has limit $0$.

(Claim 2 below records that under (D0) hypothesis (I) of Asymptotic Lower Bound for the Recentred N-Agent Cost without Uniform Control-Moment Hypotheses holds with the zero matrix as $\Pi_{0}$, so that an admissible family satisfies every hypothesis that theorem and the certificate lemma impose on a family.) Write $\mathcal{C}$ for the set of all sequences $(\mathcal{J}_{N})_{N\ge1}$ of recentred costs of admissible families, a set of real sequences (claim 3 below records that $\mathcal{C}$ is nonempty).

The approximate Kalman family. For every natural number $N\ge1$ let $h^{\mathrm{K},N}$ (written $h^{N}$ in the cited items) be the approximate Kalman policy at level $N$ of the data $l$, $m$, $\tilde{l}$, $\mathcal{A}$, $\beta$, $(U,V,\bar\beta)$, $(U_{c},\bar{L},\bar{G})$, $T$, $(S,A,P)$, $\tilde\beta$, $(\tilde{U},\bar{\tilde\beta})$, with the Riccati family $Z$ and with the zero matrix in the role of the matrix $\Pi_{0}$ of its hypothesis (H4) (claim 1 below records that the lemma applies and that its filter covariance is $\Pi$); let an $N$-agent driving system $(\Omega^{\mathrm{K}},\mathcal{F}^{\mathrm{K}},P^{\mathrm{K}})$ with $l$ states and $\tilde{l}$ observation channels, with expectation $\mathbb{E}^{\mathrm{K}}$, be given, and let a projected solution of the controlled $N$-agent dynamics on $[0,T]$ for $\beta$, $\tilde\beta$, this driving system and $h^{\mathrm{K},N}$, as in conclusion 1 of Existence of Asymptotically Optimal-Value Observation-Driven Policies, be fixed, with empirical state measure $\Sigma^{\mathrm{K}}$, state fluctuation $\mathfrak{s}^{\mathrm{K}}_{t}=\sqrt{N}(\Sigma^{\mathrm{K}}_{t}-S_{t})$ and recentred cost $\mathcal{J}^{\mathrm{K}}_{N}$ (the dependence on $N$ again suppressed; the objects named for a family of solutions above are written with the superscript $\mathrm{K}$ for this one); claim 3 below records that these objects form a family of solutions in the sense above, the Kalman family. Assume:

(DK) there are points $\mathsf{y}^{N}_{0}\in\mathbb{G}_{N}$ with $P^{\mathrm{K}}(\Sigma^{\mathrm{K}}_{0}=\mathsf{y}^{N}_{0})=1$ for every $N\ge1$ such that the real sequence $(\sqrt{N}\,|\mathsf{y}^{N}_{0}-S_{0}|)_{N\ge1}$ has limit $0$.

Notational cautions: the matrix $W_{t}$ carries a time subscript, the observation record $W$ does not; $P^{\mathrm{ag}}$, $P^{\mathrm{K}}$ are probability measures, $P$ with a time subscript is the co-state, and $P_{0}$ in the instantiation of the Riccati theorem is the name of its initial-value datum; in that instantiation $A$, $C$, $D$ are the names of the data of the Riccati theorem, whereas $A$ with a time subscript is the mean-field control of the triple; $\mathcal{A}$ is the control set, whereas $\mathcal{A}_{s}(\lambda)$ with subscript and argument is an information functional of the cited items; $K$ is the derivative bound of $(U,V,\bar\beta)$, whereas the block count of the ledger lemma, written $K$ there, is not named here; $Q_{t}$ with a time subscript is a coefficient matrix, whereas the noise majorant written $Q$ in the cited settings is not named here; $B$ is the rate bound and $\mathsf{B}_{t}$, $\mathcal{B}_{t}$ are control matrices; $R$ is the control bound and $R_{t}$ a coefficient matrix; $E_{t}$ and $\mathcal{E}_{t}$ are the state matrix, whereas the path set written $E$ and the energy written $\mathcal{E}$ in the cited settings are not named here; the roman superscript $\mathrm{K}$ marks the objects of the Kalman family and is unrelated to $K$; $\mathcal{G}_{t}$ is the observation filtration, whereas the feedback gain of The Approximate Kalman Filter and Policy for the Controlled N-Agent Dynamics is not named here; the letter $V$ in $(U,V,\bar\beta)$ is an open set, unrelated to the matrices $V_{t}$ and to $V^{*}$; $\mathcal{C}$ is a set of real sequences, unrelated to the Riccati datum $C$; $\Phi$ denotes the realized mean-field flow of the cited settings, the fundamental solution written $\Phi$ (with inverse $\Psi$) in The Approximate Kalman Filter and Policy for the Controlled N-Agent Dynamics not being named here; and (DK) is a hypothesis on the Kalman family, the existence of driving systems with prescribed deterministic initial aggregate states not being asserted here.

Then the following hold.

1. (The Kalman-policy lemma and the attainment theorem apply; the value is well defined.) Each $R_{t}$ and each $\tilde{\Theta}^{\star}_{t}$ is invertible. The hypotheses (H1)--(H4) and (C) of The Approximate Kalman Filter and Policy for the Controlled N-Agent Dynamics hold for the data named above, with the zero matrix as $\Pi_{0}$ and with $\tilde{r}=\underline{b}$ in (H3); its coefficient matrices $Q_{t}$, $V_{t}$, $R_{t}$, $\hat{F}$ are the matrices so named above, its state and control matrices are $E_{t}$ and $\mathsf{B}_{t}$, its Riccati family can be taken to be $Z$, whose matrices $W_{t}$ are those defined above, and its filter covariance is $\Pi$: the Riccati theorem applies, $\Pi$ exists and is unique with continuous entries, and every $\Pi_{t}$ is symmetric positive semidefinite. The control set $\mathcal{A}$ is convex and hypothesis (H5) of Cost Limit Along the Approximate Kalman Policy holds with $\varrho=\varrho_{\mathcal{A}}$, so Existence of Asymptotically Optimal-Value Observation-Driven Policies applies to these data; the integrand defining $V^{*}$ is continuous on $[0,T]$, and the real number $V^{*}$ defined in that theorem equals the $V^{*}$ defined above.

2. (Every admissible family satisfies the LQG lower bound.) Let a family of solutions be admissible. Then hypothesis (I) of Asymptotic Lower Bound for the Recentred N-Agent Cost without Uniform Control-Moment Hypotheses holds for it with the zero matrix as $\Pi_{0}$; the setting and hypotheses of Injection Certificates on the Trimmed Synthetic Copy for the Family of N-Agent Solutions: the Law-Transported Van Trees Certificate Hypothesis Holds under Deterministic Initial States, Uniform Observation Positivity and a C^2 Observation Extension are instantiated by the common data, (OC), (X$'$) and this family, once the auxiliary objects fixed in that setting (among them the natural number $N_{\mathrm{cl}}$, the dense sequences and the reconstruction data) are chosen, which is possible; hypothesis (MS) of Asymptotic Lower Bound for the Recentred N-Agent Cost by the Fluctuation LQG Value, under Law-Transported Injection Certificates holds for this family; every hypothesis of Asymptotic Lower Bound for the Recentred N-Agent Cost by the Fluctuation LQG Value, under Law-Transported Injection Certificates holds for the common data and this family, its $\Pi$ being the $\Pi$ above and its real number $V_{0}$ being $\int_{[0,T]}\sum_{\gamma,\delta}Z^{\gamma\delta}_{t}\Theta^{\star\gamma\delta}_{t}\,dt$; and consequently, for every real $\varepsilon>0$ there is a natural number $N_{\varepsilon}$ with

$$\mathcal{J}_{N}\ \ge\ V^{*}-\varepsilon\qquad\text{for every }N\ge N_{\varepsilon}.$$

3. (The Kalman family is admissible and attains the value.) For every $N\ge1$, $\mathbb{E}^{\mathrm{K}}[\mathfrak{s}^{\mathrm{K},\gamma}_{0}\mathfrak{s}^{\mathrm{K},\delta}_{0}]=N(\mathsf{y}^{N,\gamma}_{0}-S^{\gamma}_{0})(\mathsf{y}^{N,\delta}_{0}-S^{\delta}_{0})$ for all $\gamma,\delta\in\{1,\dots,l\}$ and $\mathbb{E}^{\mathrm{K}}[|\mathfrak{s}^{\mathrm{K}}_{0}|^{4}]=N^{2}|\mathsf{y}^{N}_{0}-S_{0}|^{4}$, so that, with the limit assumed in (DK), the initial-condition hypotheses (I1) (with the zero matrix as $\Pi_{0}$) and (I2) of Cost Limit Along the Approximate Kalman Policy hold for the Kalman family; each $\mathcal{J}^{\mathrm{K}}_{N}$ is a well-defined real number and

$$\lim_{N\to\infty}\mathcal{J}^{\mathrm{K}}_{N}\ =\ V^{*}.$$

Moreover the Kalman family satisfies (I$'$), (CB) and (D0) — the last with $\mathsf{x}^{N}_{0}=\mathsf{y}^{N}_{0}$ — and is therefore admissible; in particular $\mathcal{C}$ is nonempty and contains $(\mathcal{J}^{\mathrm{K}}_{N})_{N\ge1}$.

4. (The fluctuation LQG value is the asymptotically optimal value.) $V^{*}$ is an asymptotically optimal value of $\mathcal{C}$, and it is the only real number with this property: if a real number $v$ is an asymptotically optimal value of $\mathcal{C}$ then $v=V^{*}$. Thus $V^{*}$ is the asymptotically optimal value of the recentred $N$-agent cost over admissible families. Moreover $V^{*}$ is the minimal linear-quadratic-Gaussian cost of the fluctuation problem: for every linear-Gaussian state-observation model on $[0,T]$ with state dimension $l$, observation dimension $\tilde{l}$ and any Brownian dimension $m^{\circ}\ge1$, matched to the fluctuation LQG data as in conclusion 3 of Existence of Asymptotically Optimal-Value Observation-Driven Policies (through hypotheses (i)--(iii) of The Limiting Cost Along the Approximate Kalman Policy Is the Optimal Value of the Fluctuation LQG Problem, with control dimension $m$, control matrix assignment $t\mapsto\mathcal{B}_{t}$ and cost data $Q_{t}$, $\tfrac12V_{t}$, $R_{t}$, $\hat{F}$), the linear-quadratic-Gaussian cost $J$ formed there attains a minimum over the extended admissible controls with values in $\mathbb{R}^{m}$, and $\min J=V^{*}$.