Uniform Convergence in Probability of the N-Agent Empirical State Measure to the Unique Optimal Mean-Field Trajectory

Adopt the setting, hypotheses and notation of the convergence theorem for the optimal $N$-agent value. In particular: $(\beta_{0},\beta_{1})$ is an affine-controlled transition-rate family on $l$ states with control set $\mathcal{A}\subseteq\mathbb{R}^{m}$, with projected extension $\beta$, rate bound $B$ and state-Lipschitz constant $\Lambda_{b}$; $\tilde{\beta}$, $T>0$ and the population cost data $(L,G)$ are as there; $\Delta^{l}$ is the probability simplex, $|\cdot|$ the Euclidean norm, $\mathcal{U}_{\mathcal{A}}$ the set of $\mathcal{A}$-valued controls and $S(x_{0},\xi)$ the mean-field flow of claim 2 of the flow stability lemma; $\sigma\in\Delta^{l}$ is the initial state, and $\mathcal{M}^{*}_{\sigma}$ and $\mathcal{S}^{*}_{\sigma}$ are the set of optimal mean-field controls and the set of optimal mean-field trajectories from $\sigma$ in the sense of the definition of the optimal value, the optimal controls and the optimal trajectories of the mean-field problem; and $D$ is the deviation from the optimal set of claim 2 of the optimal-set structure lemma. For every natural number $N$ there are given the $N$-agent driving system $(\Omega^{N},\mathcal{F}^{N},P^{N})$ with expectation $\mathbb{E}^{N}$, the $\mathcal{A}$-valued policy $h^{N}$, the solution of the controlled $N$-agent dynamics with empirical state measure $\Sigma^{N}$, and the realized control $\hat{\alpha}^{N}$ of the realized-control lemma; the sequence $(\epsilon_{N})_{N\in\mathbb{N}}$ and the hypothesis that $(\Sigma^{N}_{0})_{N\in\mathbb{N}}$ converges in distribution to $Y_{\sigma}$ are as there, and $D_{N}$ is the random variable of claim 4 of that theorem, so that $D_{N}(\omega)=D(\Sigma^{N}_{0}(\omega),\hat{\alpha}^{N}(\omega))$.

Assume in addition that the set $\mathcal{S}^{*}_{\sigma}$ has exactly one element, and write $S^{*}$ for that element, so that $\mathcal{S}^{*}_{\sigma}=\{S^{*}\}$; it is a map on $[0,T]$ with $S^{*}_{t}\in\Delta^{l}$ for every $t\in[0,T]$, by claim 2 of the flow stability lemma.

For every natural number $N$ let $\Omega^{N}_{*}\in\mathcal{F}^{N}$ and $\overline{M}^{N}$ be the event and the random variable furnished by the martingale bound for the empirical state measure, applied to the $N$-th solution: thus $P^{N}(\Omega^{N}_{*})=1$, the path $t\mapsto\Sigma^{N}_{t}(\omega)$ is right-continuous at every $t\in[0,T)$ for every $\omega\in\Omega^{N}_{*}$, and

Write $\mathbf{1}_{\Omega^{N}_{*}}$ for the function on $\Omega^{N}$ equal to $1$ on $\Omega^{N}_{*}$ and to $0$ elsewhere.

For a natural number $n$ put

a finite nonempty set, where a natural number is read in $\mathbb{R}$ through its image in the real field. For natural numbers $N$ and $n$ let $g^{N}_{n}$ be the map on $\Omega^{N}$ given by

the maximum of a finite nonempty set of real numbers, and let

Then the following hold.

1. (The deviation measures the distance to the optimal trajectory.) $S(\sigma,\zeta)=S^{*}$ for every $\zeta\in\mathcal{M}^{*}_{\sigma}$. Moreover, for every $x_{0}\in\Delta^{l}$ and every $\xi\in\mathcal{U}_{\mathcal{A}}$ the supremum

exists and is equal to $D(x_{0},\xi)$.

2. (The uniform deviation is a random variable.) For all natural numbers $N$ and $n$ the map $g^{N}_{n}$ is a random variable on $(\Omega^{N},\mathcal{F}^{N},P^{N})$ with $0\le g^{N}_{n}(\omega)\le2$ for every $\omega\in\Omega^{N}$. For every natural number $N$ the supremum defining $W_{N}(\omega)$ exists for every $\omega\in\Omega^{N}$, and $W_{N}$ is a random variable with $0\le W_{N}(\omega)\le2$ for every $\omega\in\Omega^{N}$. Moreover, for every $\omega\in\Omega^{N}_{*}$ the supremum

exists and is equal to $W_{N}(\omega)$.

3. (Uniform convergence to the optimal trajectory in probability.) For every real $\varepsilon>0$ and every natural number $N$,

and the sequence

converges to $0$.