Convergence of the Optimal N-Agent Value to the Optimal Mean-Field Value and Concentration of Asymptotically Optimal Controls

Adopt the setting, hypotheses and notation of the asymptotic lower bound theorem for the $N$-agent cost. In particular: $(\beta_{0},\beta_{1})$ is an affine-controlled transition-rate family on $l$ states with control set $\mathcal{A}\subseteq\mathbb{R}^{m}$ and projected extension $\beta$, with rate bound $B$, aggregate state drift $b$ and state-Lipschitz constant $\Lambda_{b}$; $\tilde{\beta}$ is an observation-rate family on $l$ states with $\tilde{l}$ channels; $T>0$ is a real number; $(L,G)$ is population cost data on $l$ states with control dimension $m$ that is convex in the control on $\mathcal{A}$; $C$ is the bound of claim 1 of the lemma on cost data over a compact control set; $S(x_{0},\xi)$ is the mean-field flow of claim 2 of the flow stability lemma; $F$ is the mean-field cost of a control from an initial state under $(L,G)$; $\mathcal{U}_{\mathcal{A}}$ is the set of $\mathcal{A}$-valued controls; and $\rho$, $d_{\Delta}$, $X=\Delta^{l}\times\mathcal{U}_{\mathcal{A}}$ and $d_{X}$ are the metrics and the product space of the compactness lemma for the simplex, the control set and their product, where $\Delta^{l}$ is the probability simplex. Write $\mathcal{B}(X)$ for the Borel $\sigma$-algebra of $(X,d_{X})$ and $|\cdot|$ for the Euclidean norm.

The initial state is $\sigma\in\Delta^{l}$, and $J^{*}_{\sigma}$ and $\mathcal{M}^{*}_{\sigma}$ are the optimal mean-field value and the set of optimal mean-field controls from $\sigma$ in the sense of the definition of the optimal value, the optimal controls and the optimal trajectories of the mean-field problem. Let $D$ be the deviation from the optimal set furnished by claim 2 of the optimal-set structure lemma, a function on $X$.

For every natural number $N$ there are given an $N$-agent driving system $(\Omega^{N},\mathcal{F}^{N},P^{N})$, an observation-driven control policy $h^{N}$ with horizon $T$, control dimension $m$ and $\tilde{l}$ channels that is $\mathcal{A}$-valued, and a solution of the controlled $N$-agent dynamics on $[0,T]$ for $\beta$, $\tilde{\beta}$, that driving system and that policy, with empirical state measure $\Sigma^{N}$; and $\hat{\alpha}^{N}$ is the realized control attached to those data by the realized-control lemma. Write $\mathbb{E}^{N}$ for the expectation on $(\Omega^{N},\mathcal{F}^{N},P^{N})$, write $J^{N}[h]$ for the $N$-agent cost under $(L,G)$ of an observation-driven control policy $h$ with horizon $T$, control dimension $m$ and $\tilde{l}$ channels for the $N$-th driving system, and let $\mu^{N}$ be the law of the random element $(\Sigma^{N}_{0},\hat{\alpha}^{N})$ of $(X,d_{X})$ furnished by claim 5 of the realized-control lemma. Finally, $Y_{\sigma}$ is the map with constant value $\sigma$ on a probability space, a random element of $(\Delta^{l},d_{\Delta})$ by claim 1 of the lemma on convergence in distribution to a constant, and it is assumed that $(\Sigma^{N}_{0})_{N\in\mathbb{N}}$ converges in distribution to $Y_{\sigma}$.

Write $\mathcal{H}$ for the set of all $\mathcal{A}$-valued observation-driven control policies with horizon $T$, control dimension $m$ and $\tilde{l}$ channels, and for a natural number $N$ put

The set $\mathcal{J}_{N}$ is nonempty, because $h^{N}\in\mathcal{H}$. For every $h\in\mathcal{H}$, part (ii) of the existence, uniqueness and regularity theorem for the controlled $N$-agent dynamics furnishes a solution on $[0,T]$ for $\beta$, $\tilde{\beta}$, the $N$-th driving system and the policy $h$, and claim 2 of the comparison lemma for the $N$-agent system and the mean-field flow, applied to that solution, shows that $J^{N}[h]$ is a real number with $|J^{N}[h]|\le C(T+1)$; so by claim 6 of the absolute-value lemma the number $-C(T+1)$ is a lower bound of $\mathcal{J}_{N}$. Hence the infimum of $\mathcal{J}_{N}$ exists by Existence of the Infimum of a Nonempty Subset of $\mathbb{R}$ Bounded Below and is unique by Uniqueness of the Supremum and of the Infimum. The optimal $N$-agent value is the real number

Let $(\epsilon_{N})_{N\in\mathbb{N}}$ be a sequence of real numbers with $0\le\epsilon_{N}$ for every $N$, converging to $0$, and such that

Then the following hold.

1. (The optimal values are bounded reals.) For every natural number $N$ one has $|V_{N}|\le C(T+1)$ and $V_{N}\le J^{N}[h^{N}]$.

2. (Open-loop policies built from optimal mean-field controls.) The set $\mathcal{M}^{*}_{\sigma}$ is nonempty. Let $\xi^{*}\in\mathcal{M}^{*}_{\sigma}$, let $A^{*}$ be an admissible representative of $\xi^{*}$ in the sense of claim 2 of the flow stability lemma, and let $h^{A^{*}}$ be the open-loop policy determined by $A^{*}$. Then $h^{A^{*}}\in\mathcal{H}$, and the sequence $\bigl(J^{N}[h^{A^{*}}]\bigr)_{N\in\mathbb{N}}$ converges to $J^{*}_{\sigma}$.

3. (Convergence of the values.) The sequences $(V_{N})_{N\in\mathbb{N}}$ and $\bigl(J^{N}[h^{N}]\bigr)_{N\in\mathbb{N}}$ both converge to $J^{*}_{\sigma}$.

4. (Concentration on the optimal controls.) For every natural number $N$ the map $D_{N}$ on $\Omega^{N}$ defined by $D_{N}(\omega)=D\bigl(\Sigma^{N}_{0}(\omega),\hat{\alpha}^{N}(\omega)\bigr)$ is a random variable, and for every real $\varepsilon>0$ the sequence

converges to $0$.