Asymptotic Lower Bound for the N-Agent Cost and Concentration of the Limit Laws on the Optimal Mean-Field Controls

Adopt the setting, hypotheses and notation of the comparison lemma for the $N$-agent system and the mean-field flow. In particular: $(\beta_{0},\beta_{1})$ is an affine-controlled transition-rate family on $l$ states with control set $\mathcal{A}\subseteq\mathbb{R}^{m}$ and transition-rate family $\beta$, with rate bound $B$, aggregate state drift $b$ and state-Lipschitz constant $\Lambda_{b}$; $\tilde{\beta}$ is an observation-rate family on $l$ states with $\tilde{l}$ channels; $T>0$ is a real number; $(L,G)$ is population cost data on $l$ states with control dimension $m$ that is convex in the control on $\mathcal{A}$; $C$ is the bound of claim 1 of the lemma on cost data over a compact control set, and $C_{F}$ is the bound of claim 1 of the boundedness, lower-semicontinuity and attainment theorem for the mean-field cost; $S(z_{0},\xi)$, for $z_{0}\in\Delta^{l}$ and $\xi\in\mathcal{U}_{\mathcal{A}}$, is the mean-field flow of claim 2 of the flow stability lemma; $F$ is the mean-field cost of a control from an initial state under $(L,G)$, regarded as a function on $X$; $\mathcal{U}_{\mathcal{A}}$ is the set of $\mathcal{A}$-valued controls; and $\rho$, $d_{\Delta}$, $X=\Delta^{l}\times\mathcal{U}_{\mathcal{A}}$ and $d_{X}$ are the metrics and the product space of the compactness lemma for the simplex, the control set and their product, where $\Delta^{l}$ is the probability simplex. Write $\mathcal{B}(X)$ for the Borel $\sigma$-algebra of $(X,d_{X})$. By claim 1 of that compactness lemma the simplex $\Delta^{l}$ is nonempty, so applying claim 5 of the attainment theorem at any of its points shows that $\mathcal{U}_{\mathcal{A}}$ is nonempty; hence $X$ is nonempty.

Let $x_{0}\in\Delta^{l}$, and let $J^{*}_{x_{0}}$ and $\mathcal{M}^{*}_{x_{0}}$ be the optimal mean-field value and the set of optimal mean-field controls from the initial state $x_{0}$, in the sense of the definition of the optimal value, the optimal controls and the optimal trajectories of the mean-field problem.

For a natural number $k$ write $1/k$ for the multiplicative inverse of the image of $k$ in $\mathbb{R}$, which is a positive real number by claim 3 of Properties of the Canonical Map from the Natural Numbers to an Ordered Field.

For every natural number $N$ let there be given an $N$-agent driving system $(\Omega^{N},\mathcal{F}^{N},P^{N})$, an observation-driven control policy $h^{N}$ with horizon $T$, control dimension $m$ and $\tilde{l}$ channels that is $\mathcal{A}$-valued, and a solution of the controlled $N$-agent dynamics on $[0,T]$ for $\beta$, $\tilde{\beta}$, that driving system and that policy, with empirical state measure $\Sigma^{N}$; and let $\hat{\alpha}^{N}$ be the realized control attached to those data by the realized-control lemma. Write $\mathbb{E}^{N}$ for the expectation on $(\Omega^{N},\mathcal{F}^{N},P^{N})$, write $J^{N}[h^{N}]$ for the $N$-agent cost of the policy $h^{N}$, and let $\mu^{N}$ be the law of the random element $(\Sigma^{N}_{0},\hat{\alpha}^{N})$ of $(X,d_{X})$ furnished by claim 5 of the realized-control lemma.

Finally, let $(\Omega,\mathcal{F},P)$ be a probability space and let $Y_{x_{0}}:\Omega\to\Delta^{l}$ be the map with $Y_{x_{0}}(\omega)=x_{0}$ for every $\omega\in\Omega$, which is a random element of $(\Delta^{l},d_{\Delta})$ by claim 1 of the lemma on convergence in distribution to a constant. Assume that the sequence $(\Sigma^{N}_{0})_{N\in\mathbb{N}}$ converges in distribution to $Y_{x_{0}}$.

Put

$$A=\{x_{0}\}\times\mathcal{U}_{\mathcal{A}}\subseteq X .$$

Then the following hold.

1. (Laws, integrability, and the expected mean-field cost.) For every natural number $N$ the law $\mu^{N}$ is a Borel measure on $(X,d_{X})$ with $\mu^{N}(X)=1$; the function $F$ is measurable with respect to $\mathcal{B}(X)$ and the Borel $\sigma$-algebra of the real line and is integrable with respect to $\mu^{N}$; the cost $J^{N}[h^{N}]$ is a real number with $|J^{N}[h^{N}]|\le C(T+1)$; and

$$\mathbb{E}^{N}\bigl[F\bigl(\Sigma^{N}_{0},\hat{\alpha}^{N}\bigr)\bigr]=\int_{X}F\,d\mu^{N}.$$

2. (Every limit law is carried by the initial state.) The set $A$ belongs to $\mathcal{B}(X)$. There exist a strictly increasing sequence $(N_{j})_{j\in\mathbb{N}}$ of natural numbers and a Borel measure $\mu$ on $(X,d_{X})$ with $\mu(X)=1$ such that $(\mu^{N_{j}})_{j\in\mathbb{N}}$ converges weakly to $\mu$. Moreover, whenever $(N_{j})_{j\in\mathbb{N}}$ is a strictly increasing sequence of natural numbers and $\mu$ is a Borel measure on $(X,d_{X})$ with $\mu(X)=1$ such that $(\mu^{N_{j}})_{j\in\mathbb{N}}$ converges weakly to $\mu$, one has $\mu(A)=1$.

3. (Asymptotic lower bound for the costs.) The sequence $(J^{N}[h^{N}])_{N\in\mathbb{N}}$ is bounded, and

$$J^{*}_{x_{0}}\le\liminf_{N}J^{N}[h^{N}],$$

the limit inferior being that of a bounded real sequence.

4. (Limit laws of an asymptotically optimal sequence.) Assume in addition that the sequence $(J^{N}[h^{N}])_{N\in\mathbb{N}}$ converges to $J^{*}_{x_{0}}$. Then the set $\{x_{0}\}\times\mathcal{M}^{*}_{x_{0}}$ belongs to $\mathcal{B}(X)$, and whenever $(N_{j})_{j\in\mathbb{N}}$ is a strictly increasing sequence of natural numbers and $\mu$ is a Borel measure on $(X,d_{X})$ with $\mu(X)=1$ such that $(\mu^{N_{j}})_{j\in\mathbb{N}}$ converges weakly to $\mu$, one has

$$\mu\bigl(\{x_{0}\}\times\mathcal{M}^{*}_{x_{0}}\bigr)=1 .$$