Martingale Decomposition of the Empirical State Measure and the Observation Process

Adopt the setting of the controlled $N$-agent dynamics with $N$ agents, $l$ states, $\tilde{l}$ observation channels, and control dimension $m$, and let $\mathcal{A}$ be a nonempty subset of Euclidean space $\mathbb{R}^m$: a transition-rate family $\beta$ with control set $\mathcal{A}$ and rate bound $B$, an observation-rate family $\tilde{\beta}$ with rate bound $\tilde{B}$, a horizon $T>0$, an $N$-agent driving system, an observation-driven control policy $h$ which is $\mathcal{A}$-valued, and a solution on $[0,T]$ (which exists by the existence and uniqueness theorem), with empirical state measure $\Sigma_t$, observation processes $\Upsilon^\upsilon_t$, control $\alpha_t$, and system filtration $(\mathcal{F}^{\mathrm{sys}}_t)_{t\in[0,T]}$. Let $b$ be the aggregate state drift of $\beta$, let $\tilde{b}$ be the aggregate observation drift of $\tilde{\beta}$, and let $\Theta$ be the aggregate fluctuation covariance of $\beta$.

(a) (Integrands.) Write $\mathbf{1}_{\Omega_0}$ for the function equal to $1$ on the regular event $\Omega_0$ of the solution and $0$ off it. For every $\omega\in\Omega_0$, all $\gamma,\delta\in\{1,\dots,l\}$ and every $\upsilon\in\{1,\dots,\tilde{l}\}$, the paths $s\mapsto b^\gamma(\Sigma_s,\alpha_s)$, $s\mapsto\tilde{b}^\upsilon(\Sigma_s)$, and $s\mapsto\Theta^{\gamma\delta}(\Sigma_s,\alpha_s)$ are measurable on $[0,T]$ with the trace Borel $\sigma$-algebra and bounded in absolute value by $2(l-1)B$, $\tilde{B}$, and $2(l-1)B$ respectively (bounds chosen uniform in the indices, not the least possible). Consequently, for every $\omega\in\Omega$ the paths $s\mapsto\mathbf{1}_{\Omega_0}b^\gamma(\Sigma_s,\alpha_s)$, $s\mapsto\mathbf{1}_{\Omega_0}\tilde{b}^\upsilon(\Sigma_s)$ and $s\mapsto\mathbf{1}_{\Omega_0}\Theta^{\gamma\delta}(\Sigma_s,\alpha_s)$ are measurable on $[0,T]$ and bounded in absolute value by the same three constants, since off $\Omega_0$ each is identically $0$; so all Lebesgue integrals below exist at every point of $\Omega$.

(b) (Decomposition.) Define, for $\gamma\in\{1,\dots,l\}$ and $\upsilon\in\{1,\dots,\tilde{l}\}$,

Then each $(M^\gamma_t)_{t\in[0,T]}$ and each $(\tilde{M}^\upsilon_t)_{t\in[0,T]}$ is a square-integrable martingale with respect to $(\mathcal{F}^{\mathrm{sys}}_t)_{t\in[0,T]}$ (time index restricted to $[0,T]$), with $M^\gamma_0=0$ at every point of $\Omega$ and $\tilde{M}^\upsilon_0=0$ almost surely.

(c) (Covariation identities.) Write $\mathbf{1}_D$ for the function equal to $1$ on $D$ and $0$ off $D$. For all $0\le r\le t\le T$, every event $D\in\mathcal{F}^{\mathrm{sys}}_r$, all $\gamma,\delta\in\{1,\dots,l\}$, and all $\upsilon,\upsilon'\in\{1,\dots,\tilde{l}\}$, all products appearing below are integrable and