Uniform Mean-Square Bound for the Martingale Part of the Empirical State Measure

Adopt the setting of the controlled $N$-agent dynamics with $N$ agents, $l$ states and control dimension $m$, and let $\mathcal{A}$ be a nonempty subset of Euclidean space $\mathbb{R}^m$: a transition-rate family $\beta$ on $l$ states with control set $\mathcal{A}$ and rate bound $B$, an observation-rate family $\tilde{\beta}$ with $\tilde{l}$ channels, a horizon $T>0$, an $N$-agent driving system $(\Omega,\mathcal{F},P)$, an observation-driven control policy $h$ which is $\mathcal{A}$-valued, and a solution on $[0,T]$ with empirical state measure $\Sigma$, control $\alpha$ and regular event $\Omega_0$, which exists by the existence and uniqueness theorem. Let $b$ be the aggregate state drift of $\beta$ and let $M=(M^1,\dots,M^l)$ be the martingale part of the martingale decomposition of the empirical state measure, and write $\mathbf{1}_{\Omega_0}$ for the function equal to $1$ on $\Omega_0$ and $0$ off it, so that

$$M^\gamma_t=\Sigma^\gamma_t-\Sigma^\gamma_0-\int_{[0,t]}\mathbf{1}_{\Omega_0}\,b^\gamma(\Sigma_s,\alpha_s)\,ds\qquad(t\in[0,T],\ \gamma\in\{1,\dots,l\}) .$$

Let $D$ be the set of dyadic partition points of $[0,T]$ as in the supremum lemma for bounded right-continuous processes, and set $K_M=2+2(l-1)BT$.

Then there is an event $\Omega_*\in\mathcal{F}$ with $\Omega_*\subseteq\Omega_0$ and $P(\Omega_*)=1$ such that, for every $\omega\in\Omega_*$ and every $\gamma$, the path $t\mapsto\Sigma^\gamma_t(\omega)$ is right-continuous at every $t\in[0,T)$, and the path $t\mapsto M^\gamma_t(\omega)$ satisfies $|M^\gamma_t(\omega)|\le K_M$ for all $t\in[0,T]$ and is right-continuous at every $t\in[0,T)$. Moreover, for any such event $\Omega_*$, writing $\mathbf{1}_{\Omega_*}$ for the function equal to $1$ on $\Omega_*$ and $0$ elsewhere, and

$$\overline{M^\gamma}=\sup_{t\in D}\big(|M^\gamma_t|\,\mathbf{1}_{\Omega_*}\big),\qquad \overline{M}=\Big(\sum_{\gamma=1}^l\big(\overline{M^\gamma}\big)^2\Big)^{1/2},$$

each $\overline{M^\gamma}$ and $\overline{M}$ is a random variable, one has $|M_t(\omega)|\le\overline{M}(\omega)$ for every $t\in[0,T]$ and every $\omega\in\Omega_*$, where $|M_t|$ denotes the Euclidean norm of the vector $M_t=(M^1_t,\dots,M^l_t)$ whereas $|M^\gamma_t|$ above denotes the absolute value of a real number, and

$$\mathbb{E}\big[\overline{M}^{\,2}\big]\le\frac{8\,l\,(l-1)\,B\,T}{N} .$$