Compensated Counters of the Controlled N-Agent Dynamics are Square-Integrable Martingales

Adopt the setting of the controlled $N$-agent dynamics with $N$ agents, $l$ states, $\tilde{l}$ observation channels, and control dimension $m$: a transition-rate family $\beta$ with rate bound $B$, an observation-rate family $\tilde{\beta}$ with rate bound $\tilde{B}$, a horizon $T>0$, an $N$-agent driving system, an observation-driven control policy $h$, and a solution on $[0,T]$ (which exists by the existence and uniqueness theorem), with counters $N^{i,\sigma\gamma}_t$, $\tilde{N}^{i,\upsilon}_t$, consumed clock times $A^{i,\sigma\gamma}_t$, $\tilde{A}^{i,\upsilon}_t$, and system filtration $(\mathcal{F}^{\mathrm{sys}}_t)_{t\in[0,T]}$.

Call a clock label $a$ either a triple consisting of an agent index $i\in\{1,\dots,N\}$ and an ordered pair $(\sigma,\gamma)$ of distinct states, written $a=(i,\sigma\gamma)$, or a pair consisting of an agent index $i$ and an observation channel $\upsilon\in\{1,\dots,\tilde{l}\}$, written $a=(i,\upsilon)$; write $N^a_t$, $A^a_t$ for the corresponding counter and consumed clock time (that is, $N^a_t=\tilde{N}^{i,\upsilon}_t$ and $A^a_t=\tilde{A}^{i,\upsilon}_t$ when $a=(i,\upsilon)$). Define the compensated counters

Then:

(a) For every clock label $a$, the process $(M^a_t)_{t\in[0,T]}$ is a square-integrable martingale with respect to $(\mathcal{F}^{\mathrm{sys}}_t)_{t\in[0,T]}$ (with time index restricted to $[0,T]$), and $M^a_0=0$.

(b) For all clock labels $a$ and $b$, all $0\le r\le t\le T$, and every event $D\in\mathcal{F}^{\mathrm{sys}}_r$, the products $M^a_tM^b_t$ and $M^a_rM^b_r$ are integrable and

where $\mathbf{1}_D$ denotes the function equal to $1$ on $D$ and $0$ off $D$. (Integrability of the products holds by the event count bound together with the Cauchy--Schwarz inequality for the mean-square norm.)