Martingale Decomposition of the Empirical State Measure and the Observation Process

Adopt the setting of the controlled $N$-agent dynamics with $N$ agents, $l$ states, $\tilde{l}$ observation channels, and control dimension $m$: a transition-rate family $\beta$ with rate bound $B$, an observation-rate family $\tilde{\beta}$ with rate bound $\tilde{B}$, a horizon $T>0$, an $N$-agent driving system, an observation-driven control policy $h$, and a solution on $[0,T]$ (which exists by the existence and uniqueness theorem), with empirical state measure $\Sigma_t$, observation processes $\Upsilon^\upsilon_t$, control $\alpha_t$, and system filtration $(\mathcal{F}^{\mathrm{sys}}_t)_{t\in[0,T]}$. Let $b$ be the aggregate state drift of $\beta$, let $\tilde{b}$ be the aggregate observation drift of $\tilde{\beta}$, and let $\Theta$ be the aggregate fluctuation covariance of $\beta$.

(a) (Integrands.) Almost surely, for all $\gamma,\delta\in\{1,\dots,l\}$ and $\upsilon\in\{1,\dots,\tilde{l}\}$ the paths $s\mapsto b^\gamma(\Sigma_s,\alpha_s)$, $s\mapsto\tilde{b}^\upsilon(\Sigma_s)$, and $s\mapsto\Theta^{\gamma\delta}(\Sigma_s,\alpha_s)$ are measurable on $[0,T]$ with the trace Borel $\sigma$-algebra and bounded in absolute value by $2(l-1)B$, $\tilde{B}$, and $2(l-1)B$ respectively, so all Lebesgue integrals below exist.

(b) (Decomposition.) Define, for $\gamma\in\{1,\dots,l\}$ and $\upsilon\in\{1,\dots,\tilde{l}\}$,

Then each $(M^\gamma_t)_{t\in[0,T]}$ and each $(\tilde{M}^\upsilon_t)_{t\in[0,T]}$ is a square-integrable martingale with respect to $(\mathcal{F}^{\mathrm{sys}}_t)_{t\in[0,T]}$ (time index restricted to $[0,T]$), with $M^\gamma_0=0$ and $\tilde{M}^\upsilon_0=0$.

(c) (Covariation identities.) Write $\mathbf{1}_D$ for the function equal to $1$ on $D$ and $0$ off $D$. For all $0\le r\le t\le T$, every event $D\in\mathcal{F}^{\mathrm{sys}}_r$, all $\gamma,\delta\in\{1,\dots,l\}$, and all $\upsilon,\upsilon'\in\{1,\dots,\tilde{l}\}$, all products appearing below are integrable and