Uniform Convergence in Probability of the N-Agent Empirical State Measure to the Unique Optimal Mean-Field Trajectory

Adopt the setting, hypotheses and notation of the convergence theorem for the optimal $N$-agent value. In particular: $(\beta_{0},\beta_{1})$ is an affine-controlled transition-rate family on $l$ states with control set $\mathcal{A}\subseteq\mathbb{R}^{m}$, with transition-rate family $\beta$, rate bound $B$ and state-Lipschitz constant $\Lambda_{b}$; $\tilde{\beta}$, $T>0$ and the population cost data $(L,G)$ are as there; $\Delta^{l}$ is the probability simplex, $|\cdot|$ the Euclidean norm, $\mathcal{U}_{\mathcal{A}}$ the set of $\mathcal{A}$-valued controls and $S(z_{0},\xi)$ the mean-field flow of claim 2 of the flow stability lemma; $x_{0}\in\Delta^{l}$ is the initial state, and $\mathcal{M}^{*}_{x_{0}}$ and $\mathcal{S}^{*}_{x_{0}}$ are the set of optimal mean-field controls and the set of optimal mean-field trajectories from $x_{0}$ in the sense of the definition of the optimal value, the optimal controls and the optimal trajectories of the mean-field problem; and $D$ is the deviation from the optimal set of claim 2 of the optimal-set structure lemma. For every natural number $N$ there are given the $N$-agent driving system $(\Omega^{N},\mathcal{F}^{N},P^{N})$ with expectation $\mathbb{E}^{N}$, the $\mathcal{A}$-valued policy $h^{N}$, the solution of the controlled $N$-agent dynamics with empirical state measure $\Sigma^{N}$, and the realized control $\hat{\alpha}^{N}$ of the realized-control lemma; the sequence $(\eta_{N})_{N\in\mathbb{N}}$ and the hypothesis that $(\Sigma^{N}_{0})_{N\in\mathbb{N}}$ converges in distribution to $Y_{x_{0}}$ are as there, and $D_{N}$ is the random variable of claim 4 of that theorem, so that $D_{N}(\omega)=D(\Sigma^{N}_{0}(\omega),\hat{\alpha}^{N}(\omega))$.

Assume in addition that the set $\mathcal{S}^{*}_{x_{0}}$ has exactly one element, and write $S^{*}$ for that element, so that $\mathcal{S}^{*}_{x_{0}}=\{S^{*}\}$; it is a map on $[0,T]$ with $S^{*}_{t}\in\Delta^{l}$ for every $t\in[0,T]$, by claim 2 of the flow stability lemma.

For every natural number $N$ let $\Omega^{N}_{*}\in\mathcal{F}^{N}$ and $\overline{M}^{N}$ be the event and the random variable furnished by the martingale bound for the empirical state measure, applied to the $N$-th solution: thus $P^{N}(\Omega^{N}_{*})=1$, the path $t\mapsto\Sigma^{N}_{t}(\omega)$ is right-continuous at every $t\in[0,T)$ for every $\omega\in\Omega^{N}_{*}$, and

$$\mathbb{E}^{N}\bigl[(\overline{M}^{N})^{2}\bigr]\le\frac{8\,l\,(l-1)\,B\,T}{N}.$$

Write $\mathbf{1}_{\Omega^{N}_{*}}$ for the function on $\Omega^{N}$ equal to $1$ on $\Omega^{N}_{*}$ and to $0$ elsewhere.

For a natural number $n$ put

$$Q_{n}=\{0\}\cup\Bigl\{\tfrac{j}{2^{n}}\,T\;:\;j\in\{1,\dots,2^{n}\}\Bigr\}\subseteq[0,T],$$

a finite nonempty set, where a natural number is read in $\mathbb{R}$ through its image in the real field. For natural numbers $N$ and $n$ let $g^{N}_{n}$ be the map on $\Omega^{N}$ given by

$$g^{N}_{n}(\omega)=\Bigl(\max\bigl\{\,\bigl|\Sigma^{N}_{t}(\omega)-S^{*}_{t}\bigr|\;:\;t\in Q_{n}\,\bigr\}\Bigr)\,\mathbf{1}_{\Omega^{N}_{*}}(\omega),$$

the maximum of a finite nonempty set of real numbers. For every $\omega\in\Omega^{N}$ the set $\bigl\{g^{N}_{n}(\omega):n\in\mathbb{N}\bigr\}$ is a nonempty set of real numbers bounded above by $2$: the vectors $\Sigma^{N}_{t}(\omega)$ and $S^{*}_{t}$ lie in $\Delta^{l}$ and hence have norm at most $1$ by claim 1 of the simplex compactness lemma, so $|\Sigma^{N}_{t}(\omega)-S^{*}_{t}|\le2$ and therefore $0\le g^{N}_{n}(\omega)\le2$. Hence that set has a least upper bound; write

$$W_{N}(\omega)=\sup\bigl\{\,g^{N}_{n}(\omega)\;:\;n\in\mathbb{N}\,\bigr\}$$

for it.

Then the following hold.

1. (The deviation measures the distance to the optimal trajectory.) $S(x_{0},\zeta)=S^{*}$ for every $\zeta\in\mathcal{M}^{*}_{x_{0}}$. Moreover, for every $z_{0}\in\Delta^{l}$ and every $\xi\in\mathcal{U}_{\mathcal{A}}$ the supremum

$$\sup\bigl\{\,\bigl|S_{t}(z_{0},\xi)-S^{*}_{t}\bigr|\;:\;t\in[0,T]\,\bigr\}$$

exists and is equal to $D(z_{0},\xi)$.

2. (The uniform deviation is a random variable.) For all natural numbers $N$ and $n$ the map $g^{N}_{n}$ is a random variable on $(\Omega^{N},\mathcal{F}^{N},P^{N})$ with $0\le g^{N}_{n}(\omega)\le2$ for every $\omega\in\Omega^{N}$. For every natural number $N$ the supremum defining $W_{N}(\omega)$ exists for every $\omega\in\Omega^{N}$, and $W_{N}$ is a random variable with $0\le W_{N}(\omega)\le2$ for every $\omega\in\Omega^{N}$. Moreover, for every $\omega\in\Omega^{N}_{*}$ the supremum

$$\sup\bigl\{\,\bigl|\Sigma^{N}_{t}(\omega)-S^{*}_{t}\bigr|\;:\;t\in[0,T]\,\bigr\}$$

exists and is equal to $W_{N}(\omega)$.

3. (Uniform convergence to the optimal trajectory in probability.) For every real $\varepsilon>0$ and every natural number $N$,

$$P^{N}\bigl(\{\omega\in\Omega^{N}\;:\;\varepsilon\le W_{N}(\omega)\}\bigr)\;\le\;\frac{32\,l\,(l-1)\,B\,T\,e^{2\Lambda_{b}T}}{N\,\varepsilon^{2}}\;+\;P^{N}\bigl(\{\omega\in\Omega^{N}\;:\;\varepsilon/2\le D_{N}(\omega)\}\bigr),$$

and the sequence

$$\Bigl(P^{N}\bigl(\{\omega\in\Omega^{N}\;:\;\varepsilon\le W_{N}(\omega)\}\bigr)\Bigr)_{N\in\mathbb{N}}$$

converges to $0$.