Pointwise-in-Time Tracking of the Mean-Field Flow and Cost along the Realized Control of the Controlled N-Agent Dynamics

Adopt the setting, hypotheses, and notation of the comparison lemma for the $N$-agent system and the mean-field flow: the affine-controlled transition-rate family $(\beta_0,\beta_1)$ on $l$ states with control set $\mathcal{A}\subseteq\mathbb{R}^m$ and its transition-rate family $\beta$ with rate bound $B$, aggregate state drift $b$, and state-Lipschitz constant $\Lambda_b$; the observation-rate family $\tilde{\beta}$; the horizon $T>0$; the population cost data $(L,G)$, convex in the control on $\mathcal{A}$; the probability simplex $\Delta^l$; the set $\mathcal{U}_{\mathcal{A}}$ of $\mathcal{A}$-valued controls with the metric $\rho$ of the compactness lemma for the simplex, the control set and their product, formed from the fixed dense sequence of claim 1 of the weak metrizability theorem — the same sequence, hence the same metric $\rho$, being used in the boundedness, lower-semicontinuity and attainment theorem and in the realized-control lemma cited below; the mean-field flow $S(x_0,\xi)$ of claim 2 of the flow stability lemma; the mean-field cost $F(x_0,\xi)$; the $N$-agent driving system $(\Omega,\mathcal{F},P)$, the $\mathcal{A}$-valued observation-driven control policy $h$, and the solution $(\sigma^i,\Upsilon^\upsilon,\alpha)$ with regular event $\Omega_0$ and empirical state measure $\Sigma$; the realized control $\hat{\alpha}$ of the realized-control lemma; the martingale part $M=(M^1,\dots,M^l)$ of the martingale decomposition of the empirical state measure; the event $\Omega_*\subseteq\Omega_0$ of the martingale bound; and the random variable $W$ of claim 2 of the comparison lemma, so that $W(\omega)=\int_{[0,T]}L(\Sigma_t(\omega),\hat{\alpha}(t,\omega))\,dt+G(\Sigma_T(\omega))$ for $\omega\in\Omega_*$ and $J^N[h]=\mathbb{E}[W]$. Write $|\cdot|$ for the Euclidean norm, $\mathbb{E}$ for the expectation, $\exp$ for the exponential function, and $\int_{[0,t]}\cdot\,ds$ for the Lebesgue integral over the compact interval $[0,t]$, taken to be $0$ for $t=0$. For every $\omega\in\Omega$ write $S^\omega=S(\Sigma_0(\omega),\hat{\alpha}(\omega))$ for the mean-field flow started at the realized initial state and driven by the realized control; it is defined for every $\omega$ by claim 2 of the flow stability lemma, since $\Sigma_0(\omega)\in\Delta^l$ and $\hat{\alpha}(\omega)\in\mathcal{U}_{\mathcal{A}}$, and on $\Omega_*$ it is the $S^\omega$ of claim 1 of the comparison lemma. Write $\hat{b}$ for the projected drift of claim 6 of the affine-rate lemma. Set

$$K_M=2+2(l-1)BT,\qquad \Gamma=1+\Lambda_bT\exp(\Lambda_bT).$$

(LipC) (Lipschitz cost data.) Assume there are real numbers $K_L\ge0$ and $K_G\ge0$ such that $|L(\Sigma,a)-L(\Sigma',a)|\le K_L|\Sigma-\Sigma'|$ and $|G(\Sigma)-G(\Sigma')|\le K_G|\Sigma-\Sigma'|$ for all $\Sigma,\Sigma'\in\Delta^l$ and all $a\in\mathcal{A}$. This hypothesis is used in claim 3 and in the estimate of claim 4 only. (The constant $K_M$ defined above is that of the martingale bound.)

Then the following hold.

1. (Paths of the martingale part.) For every $\omega\in\Omega_0$ and every $\gamma\in\{1,\dots,l\}$ the path $t\mapsto M^\gamma_t(\omega)$ is measurable with respect to the trace Borel $\sigma$-algebra on $[0,T]$ and the Borel $\sigma$-algebra on the real line, and $|M^\gamma_t(\omega)|\le K_M$ for every $t\in[0,T]$; consequently $t\mapsto|M_t(\omega)|$ is measurable and bounded by $\sqrt{l}\,K_M$ (with $\sqrt{\cdot}$ the nonnegative square root), and the integrals $\mathcal{M}_t(\omega)=\int_{[0,t]}|M_s(\omega)|\,ds$ exist for every $t\in[0,T]$ and are nondecreasing in $t$.

2. (Pointwise-in-time tracking of the flow.) For every $\omega\in\Omega_0$ and every $t\in[0,T]$,

$$\big|\Sigma_t(\omega)-S^\omega_t\big|\ \le\ |M_t(\omega)|+\Lambda_b\exp(\Lambda_bt)\,\mathcal{M}_t(\omega).$$

3. (Tracking of the cost.) Under (LipC), for every $\omega\in\Omega_*$,

$$\big|W(\omega)-F\big(\Sigma_0(\omega),\hat{\alpha}(\omega)\big)\big|\ \le\ K_L\,\Gamma\,\mathcal{M}_T(\omega)+K_G\Big(|M_T(\omega)|+\Lambda_b\exp(\Lambda_bT)\,\mathcal{M}_T(\omega)\Big).$$

4. (Shift of the initial state.) Let $z_0\in\Delta^l$. The map $\omega\mapsto F\big(z_0,\hat{\alpha}(\omega)\big)$ is a random variable, and under (LipC), for every $\omega\in\Omega$,

$$\big|F\big(\Sigma_0(\omega),\hat{\alpha}(\omega)\big)-F\big(z_0,\hat{\alpha}(\omega)\big)\big|\ \le\ (K_LT+K_G)\exp(\Lambda_bT)\,\big|\Sigma_0(\omega)-z_0\big| .$$