Comparison of the N-Agent System with the Mean-Field Flow and Cost along the Realized Control

Let $l$ and $m$ be natural numbers with $l\ge2$ and $m\ge1$, let $(\beta_{0},\beta_{1})$ be an affine-controlled transition-rate family on $l$ states with control set $\mathcal{A}\subseteq\mathbb{R}^{m}$ and let $\beta$ be its transition-rate family, with rate bound $B$, aggregate state drift $b$ and state-Lipschitz constant $\Lambda_{b}$, so that $|b(\Sigma,a)-b(\Sigma',a)|\le\Lambda_{b}|\Sigma-\Sigma'|$ for all $\Sigma,\Sigma'$ in the probability simplex $\Delta^{l}$ and every $a\in\mathcal{A}$ by claim 4 there, where $|\cdot|$ is the Euclidean norm. As part of the data of the rate family, $\mathcal{A}$ is a nonempty compact convex subset of Euclidean space $\mathbb{R}^{m}$.

Let $\tilde{\beta}$ be an observation-rate family on $l$ states with $\tilde{l}$ channels, where $\tilde{l}$ is a natural number with $\tilde{l}\ge1$, let $T>0$ be a real number, and let $(L,G)$ be population cost data on $l$ states with control dimension $m$ that is convex in the control on $\mathcal{A}$, meaning that for every $\Sigma\in\Delta^{l}$ the function sending $a$ to $L(\Sigma,a)$ is convex on $\mathcal{A}$; these are the standing hypotheses of the boundedness, lower-semicontinuity and attainment theorem for the mean-field cost. Let $C$ be the bound of claim 1 of the lemma on cost data over a compact control set, so that $|L(\Sigma,a)|\le C$ and $|G(\Sigma)|\le C$ for every $\Sigma\in\Delta^{l}$ and every $a\in\mathcal{A}$, and let $C_{F}$ be the bound of claim 1 of the boundedness, lower-semicontinuity and attainment theorem. Let $S(x_{0},\xi)$ be the mean-field flow of claim 2 of the flow stability lemma and let $F$ be the mean-field cost of a control from an initial state under $(L,G)$. Let $\mathcal{U}_{\mathcal{A}}$ be the set of $\mathcal{A}$-valued controls and let $\rho$, $d_{\Delta}$, $X$ and $d_{X}$ be as in the compactness lemma for the simplex, the control set and their product. Write $\lambda_{[0,T]}$ for the restricted Lebesgue measure on $[0,T]$.

Let $N$ be a natural number with $N\ge1$, let $(\Omega,\mathcal{F},P)$ be an $N$-agent driving system, let $h$ be an observation-driven control policy with horizon $T$, control dimension $m$ and $\tilde{l}$ channels that is $\mathcal{A}$-valued, and let $(\sigma^{i},\Upsilon^{\upsilon},\alpha)$ be a solution of the controlled $N$-agent dynamics on $[0,T]$ for $\beta$, $\tilde{\beta}$, this driving system and this policy, with regular event $\Omega_{0}$ and empirical state measure $\Sigma$. Let $\hat{\alpha}$ be the realized control of the realized-control lemma, so that $\hat{\alpha}(\omega)\in\mathcal{U}_{\mathcal{A}}$ for every $\omega\in\Omega$, the pair $(\Sigma_{0},\hat{\alpha})$ is a random element of $(X,d_{X})$, and $\hat{\alpha}(t,\omega)=\alpha_{t}(\omega)$ for every $t\in[0,T]$ and every $\omega\in\Omega_{0}$.

Let $M$ be the martingale part of the martingale decomposition of the empirical state measure, and let $\Omega_{*}\subseteq\Omega_{0}$ and $\overline{M}$ be the event of probability $1$ and the random variable furnished by the martingale bound for the empirical state measure, so that $|M_{t}(\omega)|\le\overline{M}(\omega)$ for every $t\in[0,T]$ and every $\omega\in\Omega_{*}$, the path $t\mapsto\Sigma_{t}(\omega)$ is right-continuous on $[0,T)$ for every $\omega\in\Omega_{*}$, and $\mathbb{E}\bigl[\overline{M}^{\,2}\bigr]\le 8l(l-1)BT/N$. Let $J^{N}[h]$ be the $N$-agent cost of the policy $h$.

Then the following hold.

1. (Pathwise comparison of the flows.) Let $\omega\in\Omega_{*}$ and write $S^{\omega}=S\bigl(\Sigma_{0}(\omega),\hat{\alpha}(\omega)\bigr)$ for the mean-field flow started at the realized initial state and driven by the realized control. Then

$$\bigl|\Sigma_{t}(\omega)-S^{\omega}_{t}\bigr|\le e^{\Lambda_{b}T}\,\overline{M}(\omega)\qquad\text{for every }t\in[0,T].$$

2. (The realized mean-field cost, and finiteness of the $N$-agent cost.) The map $\omega\mapsto F\bigl(\Sigma_{0}(\omega),\hat{\alpha}(\omega)\bigr)$ is a random variable, and $\bigl|F\bigl(\Sigma_{0}(\omega),\hat{\alpha}(\omega)\bigr)\bigr|\le C_{F}$ for every $\omega\in\Omega$. Moreover, fix a point $e\in\Delta^{l}$ and let $\Sigma^{*}:[0,T]\times\Omega\to\mathbb{R}^{l}$ be the map with $\Sigma^{*}_{t}(\omega)=\Sigma_{t}(\omega)$ for $\omega\in\Omega_{*}$ and $\Sigma^{*}_{t}(\omega)=e$ for $\omega\notin\Omega_{*}$, so that $\Sigma^{*}_{t}(\omega)\in\Delta^{l}$ for every $t$ and every $\omega$. Then

$$W(\omega)=\int_{[0,T]}L\bigl(\Sigma^{*}_{t}(\omega),\hat{\alpha}(t,\omega)\bigr)\,d\lambda_{[0,T]}(t)+G\bigl(\Sigma^{*}_{T}(\omega)\bigr)$$

defines a random variable with $|W(\omega)|\le C(T+1)$ for every $\omega\in\Omega$, the $N$-agent cost $J^{N}[h]$ is a real number, and $J^{N}[h]=\mathbb{E}[W]$.

3. (Uniform comparison of the costs.) For every real $\varepsilon>0$ there is a natural number $N_{0}$, depending only on $\varepsilon$, $l$, $B$, $T$, $\Lambda_{b}$, $\mathcal{A}$, $L$ and $G$, and not on $N$, on the driving system, on the policy or on the solution, such that

$$\Bigl|J^{N}[h]-\mathbb{E}\bigl[F\bigl(\Sigma_{0},\hat{\alpha}\bigr)\bigr]\Bigr|\le\varepsilon$$

holds for every $N\ge N_{0}$, every $N$-agent driving system, every $\mathcal{A}$-valued observation-driven control policy $h$ with horizon $T$, control dimension $m$ and $\tilde{l}$ channels, and every solution of the controlled $N$-agent dynamics for those data, with $\Sigma_{0}$ and $\hat{\alpha}$ the initial empirical state measure and the realized control of that solution.