The Path-Closeness Event under the Cost Bound: Closeness of the Empirical State Measure to the Mean-Field Trajectory and of the Record-Frozen Control to the Mean-Field Control on an Event of Probability $1-O(N^{-1/2})$

Data. Adopt the setting, notation and standing hypotheses of the control- and flow-closeness lemma, and with them those of the asymptotic lower bound theorem for the recentred $N$-agent cost: the common data, among them the affine-controlled transition-rate family $(\beta_{0},\beta_{1})$ on $l$ states with compact convex control set $\mathcal{A}\subseteq\mathbb{R}^{m}$ and control bound $R=\sup_{\alpha\in\mathcal{A}}|\alpha|$, together with its Lipschitz constant $\Lambda$ (part of the datum of an affine-controlled family by its definition, and the constant from which $\Lambda_{b}$ is formed in the affine rate family lemma, though not named separately in the lower bound theorem), its transition-rate family $\beta$ with state-Lipschitz constant $\Lambda_{b}$, the observation-rate family $\tilde{\beta}$ with $\tilde{l}$ channels, the horizon $T>0$, and the stationary mean-field triple $(S,A,P)$ with $x_{0}=S_{0}$ in the probability simplex $\Delta^{l}$ (the third entry of the triple being the co-state function, whose values $P_{t}$ always carry a time subscript), so that $(S,A)$ is a mean-field trajectory pair for $\beta$ with horizon $T$, $S$ takes its values in $\Delta^{l}$, and the components of $S$ and of $A$ are continuous on $[0,T]$, hence measurable with respect to the trace Borel $\sigma$-algebra $\mathcal{B}_{[0,T]}$ by claim 4 of the measurability toolkit; the family, indexed by the natural numbers $N\ge1$, of solutions of the controlled $N$-agent dynamics on $[0,T]$, the $N$-th of them carried by the $N$-agent driving system $(\Omega,\mathcal{F},P)$ for the $\mathcal{A}$-valued observation-driven control policy $h=h^{(N)}$, with regular event $\Omega_{0}$, empirical state measure $\Sigma$, observation filtration $(\mathcal{G}_{t})_{t\in[0,T]}$ (a filtration of sub-$\sigma$-algebras of $\mathcal{F}$), state fluctuation $\mathfrak{s}_{t}=\sqrt{N}(\Sigma_{t}-S_{t})$ for the trajectory pair $(S,A)$, and $\kappa_{0}=1+\mathbb{E}[|\mathfrak{s}_{0}|^{4}]$, the superscript $N$ being suppressed and the probability of the driving system written $P$ as there; the hypotheses (I$'$) with the bound $\kappa^{\sharp}$ and (CB) with the bound $\mathcal{J}^{\sharp}$; the realized control $\hat{\alpha}$ of the realized-control lemma, formed as in claim 2 there from a fixed family $(\tau_{j},\upsilon_{j})_{j\ge1}$ furnished by its claim 1; the mean-field flow $S(z_{0},\xi)$ of claim 2 of the flow stability lemma, defined for $z_{0}\in\Delta^{l}$ and an $\mathcal{A}$-valued control $\xi$ (the two-argument $S_{t}(\cdot,\cdot)$, as distinct from the trajectory $S_{t}$ of the triple), and the realized mean-field flow $\Phi_{t}(\omega)=S_{t}(x_{0},\hat{\alpha}(\omega))$; and, from claims 3 and 4 of the control- and flow-closeness lemma, the parameter vector $\pi_{\bullet}$, the numbers $\mathcal{Z}^{\sharp}\ge0$ and $N_{1}$, the constants $C_{\mathrm{ctl}}$, $C_{\mathrm{flw}}$ and $C_{\mathrm{esc}}$, and, for every real $\varepsilon_{\flat}$ with $0<\varepsilon_{\flat}\le1$ and every natural number $N_{\flat}\ge N_{1}$ with $\varepsilon_{\flat}N_{\flat}\ge C_{\mathrm{esc}}$, the events $\Omega^{\flat}_{N}(\varepsilon_{\flat})$ ($N\ge N_{\flat}$) of claim 4 there, written with their dependence on $\varepsilon_{\flat}$ made explicit.

Adopt further, for the $N$-th solution, its observation record $W$ with values in the observation record space $(\mathbf{R},\mathcal{R})$ with horizon $T$ and $\tilde{l}$ channels, the record-frozen control paths $a^{r}$ ($r\in\mathbf{R}$) of the policy $h^{(N)}$, and the control discrepancy

of claim 2 of the record-frozen closeness-set lemma, formed with the comparison control $A$ of the stationary triple; the setting of that lemma is instantiated by the $N$-th solution as verified at the start of the proof, its comparison path being $S^{*}=S$ with bound $K^{*}=1$ and its path data ($E$ and the derived objects) being arbitrary, say $E=\{0_{\mathbb{R}^{l}}\}$ with $0_{\mathbb{R}^{l}}$ the origin, since they enter neither of its claims 1 and 2.

Adopt finally, from the pre-stopping envelope lemma, the constants $c_{M}$, $\kappa_{T}$ and

which are determined by the common data (only the constant $c_{Q}$ defined by this display is used below), and the noise majorant

of the $N$-th solution, the object so named in the noise-majorant lemma, in which $\overline{M}=\bigl(\sum_{\gamma=1}^{l}(\overline{M}^{\gamma})^{2}\bigr)^{1/2}$ with $\overline{M}^{\gamma}=\sup_{t\in D}(\mathbf{1}_{\Omega_{0}}|M^{\gamma}_{t}|)$ the supremum random variable of the supremum lemma applied to the process $(\mathbf{1}_{\Omega_{0}}M^{\gamma}_{t})_{t\in[0,T]}$, $D$ being the dyadic partition points of $[0,T]$ and $M=(M^{1},\dots,M^{l})$ the martingale part of the martingale decomposition of the empirical state measure, and $I$ is the random variable of part (b) of the restricted-moments lemma for the martingale part; the envelope lemma is applied in the instance of its setting determined by the present data with $S^{*}=S$, with the trajectory pair $(S,A)$ of the stationary triple, and with its barriers $\varepsilon_{Y}$, $c_{\mathcal{E}}$, $\delta$ and $\theta_{\mathrm{out}}$ all given the value $1$; none of these barriers enters $Q$ or $c_{Q}$, and none survives in the conclusions used below once the tail bound of its claim 4 is rewritten in terms of $Q=\overline{\mathfrak{b}}-\varepsilon_{Y}$.

Here $\mathbb{E}$ is the expectation, $|\cdot|$ the Euclidean norm, $\int_{[0,t]}\cdot\,du$ the Lebesgue integral over a compact interval, $\exp$ and $e^{x}$ the exponential function, and $\sqrt{x}=x^{1/2}$ the nonnegative square root of a real $x\ge0$; for a natural number $N$ and a real $\theta>0$ we write $N^{1/4}=\sqrt{\sqrt{N}}$, $N^{-1/2}=1/\sqrt{N}$, $N^{-1/4}=1/N^{1/4}$, $N^{-1}=1/N$, $N^{-2}=1/N^{2}$ and $\theta^{-4}=1/\theta^{4}$, so that $(N^{1/4})^{2}=\sqrt{N}$ and $(N^{1/4})^{4}=N$. Notational cautions: the letter $Q$ denotes only the noise majorant, never a matrix of the Riccati data; the finite set $E$ of the record-frozen closeness-set lemma is unrelated to the matrices $E_{t}$ of the completion-of-squares data, and $R$ is the control bound, not the matrix $R_{t}$; $W$ denotes only the observation record, the cost random variable of the pathwise tracking lemma, the number $W^{\sharp}(\pi)$ of the lower bound theorem and the domain $W$ of the cost extension in the definition of a stationary mean-field triple playing no role here; $I$ is the random variable of the restricted-moments lemma, unrelated to the clipped-out time of the extended good-set lemma; the barrier written $\delta$ in the extended good-set lemma is unrelated to state indices; $K$ (the number of blocks) is unrelated to $K^{*}$, to the constants $K_{1},K_{2}$ and to the observation-event count $K_{t}$; the record-space reference measure, written $\varrho$ in the record-frozen closeness-set lemma, is unrelated to the component $\varrho$ of a parameter vector; and the set $D$ of dyadic partition points is unrelated to the leave events $D_{0},\dots,D_{K-1}$ of the control- and flow-closeness lemma.

Then the following hold.

1. (The noise majorant.) For every $N\ge1$: $Q$ is a random variable with

for every real $\theta>0$, $P(Q\ge\theta)\le c_{Q}\,\kappa^{\sharp}\,\theta^{-4}N^{-2}$; and for every $\omega\in\Omega_{0}$ and every $t\in[0,T]$,

2. (The closeness event.) Let $N_{2}$ be a natural number with $N_{2}\ge N_{1}$ and $N_{2}\ge C_{\mathrm{esc}}^{2}$ (such a number exists), and put

For every $N\ge N_{2}$, claim 4 of the control- and flow-closeness lemma applies to the $N$-th solution with $\varepsilon_{\flat}=N^{-1/2}$ and $N_{\flat}=N$; put $\Omega^{\flat}_{N}=\Omega^{\flat}_{N}(N^{-1/2})$ and

Then $\Omega^{\mathrm{cl}}_{N}$ is an event,

for every $\omega\in\Omega^{\mathrm{cl}}_{N}$ and every $t\in[0,T]$,

and for every $\omega\in\Omega^{\mathrm{cl}}_{N}$,

The constants $C_{\mathrm{flw}}$, $C_{\mathrm{ctl}}$ and $c_{Q}$ entering $\varepsilon_{S}(N)$, $\varepsilon_{\mathrm{ctl}}(N)$ and the probability bound are determined by the common data and the bounds $\kappa^{\sharp}$, $\mathcal{J}^{\sharp}$, and a threshold $N_{2}$ as in claim 2 may be chosen from the same data; all are the same for every $N$. The two tolerances are of order $N^{-1/4}$ and the exceptional probability is of order $N^{-1/2}$. The first bound of claim 2 concerns the empirical state measure itself, not only the realized mean-field flow, and the third expresses the control condition through the record alone.