Proof of Pointwise-in-Time Tracking of the Mean-Field Flow and Cost along the Realized Control of the Controlled N-Agent Dynamics

Throughout we use linearity and monotonicity of the Lebesgue integral and the fact that a bounded measurable function on a compact interval is integrable. Points of $\Delta^l$ have Euclidean norm at most $1$, so $|\Sigma-\Sigma'|\le2$ for $\Sigma,\Sigma'\in\Delta^l$.

Claim 1. Fix $\omega\in\Omega_0$. By condition 1 of the definition of a solution, each state path $t\mapsto\sigma^i_t(\omega)$ is constant on each of finitely many intervals partitioning $[0,T]$, so each occupation indicator $t\mapsto\eta^{i,\gamma}_t(\omega)$, and hence $t\mapsto\Sigma^\gamma_t(\omega)=\frac{1}{N}\sum_{i}\eta^{i,\gamma}_t(\omega)$, is a finite linear combination of indicator functions of intervals; intervals belong to the trace Borel $\sigma$-algebra, so these paths are measurable by measurability of sums and scalar multiples of measurable functions (indicators of measurable sets being measurable), and they take values in $[0,1]$. The path $s\mapsto b^\gamma(\Sigma_s(\omega),\alpha_s(\omega))$ is measurable and bounded in absolute value by $2(l-1)B$ by part (a) of the martingale decomposition theorem, so by part (i) of the integration-by-parts lemma (applied with that path as its first integrand, the second integrand and both initial values taken to be $0$; every later use of part (i) in this proof is of the same form) the map $t\mapsto\int_{[0,t]}b^\gamma(\Sigma_s(\omega),\alpha_s(\omega))\,ds$ is continuous on $[0,T]$, hence measurable by claim 3 of the toolkit, and it is bounded in absolute value by $2(l-1)BT$ by monotonicity. Since $\mathbf{1}_{\Omega_0}(\omega)=1$, part (b) of the martingale decomposition theorem gives $M^\gamma_t(\omega)=\Sigma^\gamma_t(\omega)-\Sigma^\gamma_0(\omega)-\int_{[0,t]}b^\gamma(\Sigma_s(\omega),\alpha_s(\omega))\,ds$, a measurable function of $t$ with $|M^\gamma_t(\omega)|\le1+2(l-1)BT\le K_M$. The path $t\mapsto|M_t(\omega)|$ is a continuous function of the measurable component paths, hence measurable by measurability of continuous functions of measurable maps, and $|M_t(\omega)|\le\sqrt{l}\,K_M$. Thus $\mathcal{M}_t(\omega)$ exists for every $t$; and for $r\le t$ we have $\mathcal{M}_r(\omega)\le\mathcal{M}_t(\omega)$, because by claim 2 of the toolkit both integrals are integrals over $\mathbb{R}$ of zero extensions, that of the restriction to $[0,r]$ being dominated pointwise by that of the restriction to $[0,t]$.

Claim 2. Fix $\omega\in\Omega_0$ and write $\xi=\hat{\alpha}(\omega)\in\mathcal{U}_{\mathcal{A}}$ and $u=\hat{\alpha}(\cdot,\omega)$, an admissible representative of $\xi$ by claim 3 of the realized-control lemma, with $u(s)=\alpha_s(\omega)$ for every $s\in[0,T]$ by claim 2 of the same lemma since $\omega\in\Omega_0$. By claim 2 of the flow stability lemma, $S^\omega=S(\Sigma_0(\omega),\xi)$ is the map furnished by claim 1 of the existence and uniqueness theorem for the initial value $\Sigma_0(\omega)$ and the control $u$: it is continuous, takes values in $\Delta^l$, and satisfies $S^{\omega,\gamma}_t=\Sigma^\gamma_0(\omega)+\int_{[0,t]}\hat{b}^\gamma(S^\omega_s,u(s))\,ds$ for all $t$ and $\gamma$, where $\hat{b}(x,a)=b(x,a)$ for $x\in\Delta^l$ by claim 6 of the affine-rate lemma. Subtracting this from the decomposition of Claim 1 and writing $y_t=\Sigma_t(\omega)-S^\omega_t$, we get, for every $t$ and $\gamma$,

$$y^\gamma_t=M^\gamma_t(\omega)+\int_{[0,t]}\Big(b^\gamma(\Sigma_s(\omega),\alpha_s(\omega))-b^\gamma(S^\omega_s,\alpha_s(\omega))\Big)ds .$$

The integrand vector $s\mapsto b(\Sigma_s,\alpha_s)-b(S^\omega_s,\alpha_s)$ has bounded measurable components (the first summand is measurable as in Claim 1; the second by the composition lemma applied to $b^\gamma$, which is sequentially continuous on $\Delta^l\times\mathcal{A}$ by the two Lipschitz bounds of claim 4 of the affine-rate lemma, and to $s\mapsto(S^\omega_s,u(s))$, whose components are measurable because $S^\omega$ is continuous and $u$ is a square-integrable, hence measurable, path by claim 3 of the realized-control lemma), so by the norm bound for vector-valued integrals and the state-Lipschitz bound of claim 4 of the affine-rate lemma,

$$|y_t|\le|M_t(\omega)|+\int_{[0,t]}\Lambda_b\,|y_s|\,ds\qquad(t\in[0,T]) .\qquad(\ast)$$

Here $s\mapsto|y_s|$ is measurable (a continuous function of measurable components) and bounded by $2$. Put $v(t)=\int_{[0,t]}|y_s|\,ds$; by part (i) of the integration-by-parts lemma $v$ is continuous on $[0,T]$. Integrating $(\ast)$ over $[0,t]$ and using monotonicity gives $v(t)\le\mathcal{M}_t(\omega)+\Lambda_b\int_{[0,t]}v(s)\,ds$ for every $t$. Now fix $t\in(0,T]$. For $s\in[0,t]$, Claim 1 gives $\mathcal{M}_s(\omega)\le\mathcal{M}_t(\omega)$, hence

$$v(s)\le\mathcal{M}_t(\omega)+\Lambda_b\int_{[0,s]}v(r)\,dr\qquad(0\le s\le t).$$

The restriction of $v$ to $[0,t]$ is continuous by claim 1 of restriction stability, and for a continuous function the Riemann and Lebesgue integrals over $[0,s]$ agree by claim 3 of the toolkit. Hence Gronwall's lemma on the interval $[0,t]$ — with, in the notation of that lemma, its function taken to be $v|_{[0,t]}$ and its two constants taken to be $\mathcal{M}_t(\omega)$ and $\Lambda_b\ge0$; the letters $u$ and $b$ retain their meanings above — yields $v(s)\le\mathcal{M}_t(\omega)\exp(\Lambda_bs)$ for $s\in[0,t]$, in particular $v(t)\le\mathcal{M}_t(\omega)\exp(\Lambda_bt)$. Inserting this into $(\ast)$ proves Claim 2 for $t\in(0,T]$; for $t=0$ both sides vanish, since $y_0=0$ and $M_0=0$.

Claim 3. Let $\omega\in\Omega_*$. Since $\Omega_*\subseteq\Omega_0$, the map $\Sigma^*$ of claim 2 of the comparison lemma satisfies $\Sigma^*_t(\omega)=\Sigma_t(\omega)$ for all $t$, so $W(\omega)=\int_{[0,T]}L(\Sigma_t(\omega),u(t))\,dt+G(\Sigma_T(\omega))$ with $u=\hat{\alpha}(\cdot,\omega)$ as in Claim 2. By the definition of the mean-field cost of a control with the admissible representative $u$, and the generalized mean-field cost, $F(\Sigma_0(\omega),\hat{\alpha}(\omega))=\int_{[0,T]}L(S^\omega_t,u(t))\,dt+G(S^\omega_T)$. Both running-cost integrands are integrable: they are bounded by the constant $C$ of the comparison lemma, and measurable by measurability of sequentially continuous functions of measurable maps, $L$ being sequentially continuous by condition 1 of the definition of population cost data and the paths $\Sigma_\cdot(\omega)$, $S^\omega$, $u$ having measurable components (Claims 1 and 2). So by (LipC), linearity, and monotonicity,

$$\big|W(\omega)-F(\Sigma_0(\omega),\hat{\alpha}(\omega))\big|\le K_L\int_{[0,T]}|y_t|\,dt+K_G|y_T| ,$$

with $y$ as in Claim 2. The function $t\mapsto\mathcal{M}_t(\omega)$ is continuous on $[0,T]$ (part (i) of the integration-by-parts lemma), so $t\mapsto\exp(\Lambda_bt)\mathcal{M}_t(\omega)$ is continuous, hence integrable, and bounded by $\exp(\Lambda_bT)\mathcal{M}_T(\omega)$ by Claim 1. Integrating the bound of Claim 2 therefore gives $\int_{[0,T]}|y_t|\,dt\le\mathcal{M}_T(\omega)+\Lambda_bT\exp(\Lambda_bT)\mathcal{M}_T(\omega)=\Gamma\,\mathcal{M}_T(\omega)$, while Claim 2 at $t=T$ gives $|y_T|\le|M_T(\omega)|+\Lambda_b\exp(\Lambda_bT)\mathcal{M}_T(\omega)$. Substituting proves Claim 3.

Claim 4. Measurability. By claim 4 of the boundedness, lower-semicontinuity and attainment theorem, the function $\xi\mapsto F(z_0,\xi)$ is lower semicontinuous on $\mathcal{U}_{\mathcal{A}}$ for the metric $\rho$. By claim 3 of the sublevel-set characterization (with $A=X=\mathcal{U}_{\mathcal{A}}$), for every real $c$ the sublevel set $\{\xi\in\mathcal{U}_{\mathcal{A}}:F(z_0,\xi)\le c\}$ is closed in $(\mathcal{U}_{\mathcal{A}},\rho)$, so its complement is open and it belongs to the Borel $\sigma$-algebra of $(\mathcal{U}_{\mathcal{A}},\rho)$, a $\sigma$-algebra being closed under complements. By claim 5 of the realized-control lemma, $\hat{\alpha}$ is a random element of $(\mathcal{U}_{\mathcal{A}},\rho)$, so $\{\omega:F(z_0,\hat{\alpha}(\omega))\le c\}=\hat{\alpha}^{-1}\big(\{\xi:F(z_0,\xi)\le c\}\big)\in\mathcal{F}$ for every real $c$, and hence also its complement $\{\omega:F(z_0,\hat{\alpha}(\omega))>c\}\in\mathcal{F}$; by the generator criterion of the definition of a measurable function, the open rays generating the Borel $\sigma$-algebra of the real line by the Borel generator lemma, the map $\omega\mapsto F(z_0,\hat{\alpha}(\omega))$ is measurable, that is, a random variable.

The estimate. Fix $\omega\in\Omega$, and let $\xi=\hat{\alpha}(\omega)$ and $u=\hat{\alpha}(\cdot,\omega)$, an admissible representative of $\xi$ by claim 3 of the realized-control lemma (this holds at every $\omega\in\Omega$). Apply claim 4 of the flow stability lemma with $x_0=\Sigma_0(\omega)$, $x_0'=z_0$, and $\xi'=\xi$: the functionals of claim 3 there satisfy $g^\gamma_r(\xi')=\langle\xi-\xi,w^{\gamma,r}\rangle_{L^2}=0$, so the slack constant of claim 4 (written $G$ in that lemma, a letter that here denotes the terminal cost) may be taken to be $0$, and $|S_t(z_0,\xi)-S_t(\Sigma_0(\omega),\xi)|\le\exp(\Lambda_bT)\,|z_0-\Sigma_0(\omega)|$ for every $t\in[0,T]$. By the definition of $F$ with the representative $u$ for both initial states,

$$F(\Sigma_0(\omega),\xi)-F(z_0,\xi)=\int_{[0,T]}\Big(L\big(S_t(\Sigma_0(\omega),\xi),u(t)\big)-L\big(S_t(z_0,\xi),u(t)\big)\Big)dt+G\big(S_T(\Sigma_0(\omega),\xi)\big)-G\big(S_T(z_0,\xi)\big),$$

and (LipC) with monotonicity bounds the absolute value of the right-hand side by $K_LT\exp(\Lambda_bT)|\Sigma_0(\omega)-z_0|+K_G\exp(\Lambda_bT)|\Sigma_0(\omega)-z_0|$, which is the asserted estimate. $\blacksquare$