Uniform Mean-Square Bound for the Martingale Part of the Empirical State Measure

Adopt the setting of the controlled $N$-agent dynamics: a transition-rate family $\beta$ on $l$ states with control dimension $m$ and rate bound $B$, an observation-rate family $\tilde{\beta}$ with $\tilde{l}$ channels, a horizon $T>0$, an $N$-agent driving system $(\Omega,\mathcal{F},P)$, an observation-driven control policy $h$, and a solution on $[0,T]$ with empirical state measure $\Sigma$, control $\alpha$ and regular event $\Omega_0$, which exists by the existence and uniqueness theorem. Let $b$ be the aggregate state drift of $\beta$ and let $M=(M^1,\dots,M^l)$ be the martingale part of the martingale decomposition of the empirical state measure, so that

Let $D$ be the set of dyadic partition points of $[0,T]$ as in the supremum lemma for bounded right-continuous processes, and set $K_M=2+2(l-1)BT$.

Then there is an event $\Omega_*\in\mathcal{F}$ with $\Omega_*\subseteq\Omega_0$ and $P(\Omega_*)=1$ such that, for every $\omega\in\Omega_*$ and every $\gamma$, the path $t\mapsto\Sigma^\gamma_t(\omega)$ is right-continuous at every $t\in[0,T)$, and the path $t\mapsto M^\gamma_t(\omega)$ satisfies $|M^\gamma_t(\omega)|\le K_M$ for all $t\in[0,T]$ and is right-continuous at every $t\in[0,T)$. Writing $\mathbf{1}_{\Omega_*}$ for the function equal to $1$ on $\Omega_*$ and $0$ elsewhere, and

each $\overline{M^\gamma}$ and $\overline{M}$ is a random variable, one has $|M_t(\omega)|\le\overline{M}(\omega)$ for every $t\in[0,T]$ and every $\omega\in\Omega_*$, and