Proof of Convergence of the N-Agent Cost to the Mean-Field Cost under an Open-Loop Control

Claim 1. Let $\Omega_*$ be the event of probability $1$ from the tracking proposition, contained in the regular event, and fix $\omega\in\Omega_*$. By the open-loop policy lemma, $\alpha_t(\omega)=A_t$ for every $t\in[0,T]$, and $A_t\in\mathcal{A}$ while $\Sigma_t(\omega)$ and $S_t$ lie in $\Delta^l$. Hence, by the boundedness and uniform continuity lemma,

Consequently the random variable

whose expectation is the $N$-agent cost $J^N[h^A]$, satisfies $|V|\le(T+1)C$ at every point of $\Omega_*$. At every point of $\Omega$ it satisfies $V\ge-(TC_L+C_G)$, where $C_L$ and $C_G$ are the lower bounds belonging to the population cost data. Since $P(\Omega_*)=1$, the one-sided version of the passage of almost sure inequalities to expectations, applied with the constant random variable $U=(T+1)C$, shows that $J^N[h^A]$ is a real number with $|J^N[h^A]|\le(T+1)C$.

By the tracking proposition, $|\Sigma_t(\omega)-S_t|\le\Psi(\omega)$ for every $t\in[0,T]$. If $\Psi(\omega)\le\delta$ then the uniform continuity clause gives

for every $t$; and in all cases these differences are at most $2C$. Writing $\mathbf{1}\{\Psi>\delta\}$ for the function equal to $1$ where $\Psi>\delta$ and $0$ elsewhere, we therefore have, at every $\omega\in\Omega_*$,

by linearity and monotonicity of the integral, and

Adding the two displays and using the triangle inequality, the random variable $V$ satisfies $\big|V-J^{MF}[(S),(A)]\big|\le(T+1)\big(\varepsilon+2C\,\mathbf{1}\{\Psi>\delta\}\big)$ at every $\omega\in\Omega_*$, the number $J^{MF}[(S),(A)]$ being exactly the sum of the two mean-field terms by the definition of the generalized mean-field cost. Since $P(\Omega_*)=1$ and $J^N[h^A]=\mathbb{E}[V]$, the one-sided version of the passage of almost sure inequalities to expectations, applied to the random variable $V-J^{MF}[(S),(A)]$, which is bounded below at every point of $\Omega$, and to the bounded random variable $U=(T+1)\big(\varepsilon+2C\,\mathbf{1}\{\Psi>\delta\}\big)$, gives

the last equality by linearity of the integral, the expectation of the indicator of an event being its probability. Finally, $\Psi^2$ is a nonnegative random variable, so Markov's inequality gives

which yields the stated bound.

Claim 2. Let $\eta>0$. Choose $\varepsilon=\eta/(2(T+1))$ and let $\delta>0$ be as in the boundedness and uniform continuity lemma for this $\varepsilon$; note that $C$ and $\delta$ do not depend on $N$. By the tracking proposition,

where $\Psi_N$ is the random variable of that proposition for the $N$-th system. Both terms in the bracket converge to $0$ as $N$ increases, the first by hypothesis, so $\mathbb{E}[\Psi_N^2]$ converges to $0$. Choose $N_0$ such that $(T+1)\,2C\,\mathbb{E}[\Psi_N^2]/\delta^2<\eta/2$ for every $N\ge N_0$. Then Claim 1 gives

As $\eta>0$ was arbitrary, $J^N[h^A]$ converges to $J^{MF}[(S),(A)]$. $\blacksquare$