Stopped Covariation Identities for the Martingale Part of the Empirical State Measure

Adopt the setting of the martingale decomposition theorem for the controlled $N$-agent dynamics with $N$ agents, $l$ states, $\tilde{l}$ observation channels, and control dimension $m$: a transition-rate family $\beta$ with control set $\mathcal{A}$, a nonempty subset of Euclidean space $\mathbb{R}^m$, and rate bound $B$, an observation-rate family $\tilde{\beta}$ with rate bound $\tilde{B}$, a horizon $T>0$, an $N$-agent driving system $(\Omega,\mathcal{F},P)$, an observation-driven control policy $h$ which is $\mathcal{A}$-valued, and a solution on $[0,T]$ with regular event $\Omega_0$, empirical state measure $\Sigma_t$ with components $\Sigma^\gamma_t$, control $\alpha_t$, consumed clock times $\mathcal{T}^{i,\sigma\gamma}_t$, and system filtration $(\mathcal{F}^{\mathrm{sys}}_t)_{t\in[0,T]}$. Let $b$ be the aggregate state drift of $\beta$, let $\Theta$ be the aggregate fluctuation covariance of $\beta$, and for $\gamma\in\{1,\dots,l\}$ let $M^\gamma=(M^\gamma_t)_{t\in[0,T]}$ be the process of part (b) of the martingale decomposition theorem,

Write $\mathbb{E}$ for the expectation, $\mathbf{1}_A$ for the function equal to $1$ on $A$ and $0$ off $A$, and set $K_M=1+2(l-1)BT$. State labels $\gamma,\delta,\sigma,\gamma'$ range over $\{1,\dots,l\}$ and agent labels $i$ over $\{1,\dots,N\}$. For $0\le r\le t\le T$, $\int_{[r,t]}\cdot\,ds$ denotes the Lebesgue integral with respect to the restricted Lebesgue measure on $[r,t]$ when $r<t$, and is $0$ when $r=t$; likewise $\int_{[0,0]}\cdot\,ds=0$. Stopping times are those of $(\mathcal{F}^{\mathrm{sys}}_t)_{t\in[0,T]}$; for a family $X=(X_t)_{t\in[0,T]}$ of real-valued functions on $\Omega$ and a function $\rho:\Omega\to[0,T]$ — in particular any stopping time, or $\min(t,\tau)$ below, which is one — $X_\rho$ denotes the sampled function; and for a stopping time $\tau$ and $s\in[0,T]$, $\mathbf{1}_{\{s<\tau\}}$ denotes the value at time $s$ of the pre-stopping-time indicator of $\tau$, the function equal to $1$ on $\{\omega\in\Omega:s<\tau(\omega)\}$ and $0$ off it. Progressive measurability is with respect to $(\mathcal{F}^{\mathrm{sys}}_t)_{t\in[0,T]}$ unless another filtration is named. A path $u\mapsto X_u(\omega)$ on $[0,T]$ is called right-continuous at $t\in[0,T)$ in the $\varepsilon$-$\eta$ sense if for every $\varepsilon>0$ there is $\eta>0$ with $|X_s(\omega)-X_t(\omega)|\le\varepsilon$ whenever $t\le s\le\min(t+\eta,T)$ (the sense of the supremum lemma for bounded right-continuous processes), and right-continuous in the sequential sense if for every $t\in[0,T]$ and every sequence $(s_j)_{j\in\mathbb{N}}$ in $[t,T]$ converging to $t$, the sequence $(X_{s_j}(\omega))_{j\in\mathbb{N}}$ converges to $X_t(\omega)$ (the sense of the progressive measurability toolkit). Throughout, a real-valued function on a subinterval $I$ of the real numbers $\mathbb{R}$ is called continuous on $I$ when it is continuous relative to $I$, both $I$ and the codomain $\mathbb{R}$ carrying the metric of the real line.

Then the following hold for all $\gamma,\delta\in\{1,\dots,l\}$.

1. (Compensator representation and regularity.) For every $t\in[0,T]$ the Lebesgue integral

exists at every point of $\Omega$ (by part (a) of the decomposition theorem), and at every point of $\Omega$

where the outer sum runs over ordered pairs $(\sigma,\gamma')$ of state labels with $\sigma\neq\gamma'$ and $\mathbf{1}_{\{\cdot\}}$ is $1$ if the subscripted condition holds and $0$ otherwise. Consequently the family $\mathcal{Q}^{\gamma\delta}=(\mathcal{Q}^{\gamma\delta}_t)_{t\in[0,T]}$ is progressively measurable, $|\mathcal{Q}^{\gamma\delta}_t(\omega)-\mathcal{Q}^{\gamma\delta}_r(\omega)|\le 2(l-1)B\,(t-r)$ for all $0\le r\le t\le T$ and every $\omega\in\Omega$ (in particular $|\mathcal{Q}^{\gamma\delta}_t|\le 2(l-1)BT$ everywhere), and every path of $\mathcal{Q}^{\gamma\delta}$ is continuous on $[0,T]$.

2. (Bounds and right-continuity of the martingale part.) $|M^\gamma_t(\omega)|\le K_M$ for every $t\in[0,T]$ and every $\omega\in\Omega$. For every $\omega\in\Omega_0$, the paths $t\mapsto\Sigma^\gamma_t(\omega)$ and $t\mapsto M^\gamma_t(\omega)$ are right-continuous at every $t\in[0,T)$ in the $\varepsilon$-$\eta$ sense. The families $\mathbf{1}_{\Omega_0}\Sigma^\gamma=(\mathbf{1}_{\Omega_0}\Sigma^\gamma_t)_{t\in[0,T]}$ and $\mathbf{1}_{\Omega_0}M^\gamma=(\mathbf{1}_{\Omega_0}M^\gamma_t)_{t\in[0,T]}$ are progressively measurable, and every path of each is right-continuous in the sequential sense.

3. (The product martingales.) The family $Y^{\gamma\delta}=(Y^{\gamma\delta}_t)_{t\in[0,T]}$ defined by

is a square-integrable martingale with respect to $(\mathcal{F}^{\mathrm{sys}}_t)_{t\in[0,T]}$ (time index restricted to $[0,T]$), with $|Y^{\gamma\delta}_t(\omega)|\le K_M^2+2(l-1)BT$ for every $t\in[0,T]$ and every $\omega\in\Omega$, and for every $\omega\in\Omega_0$ the path $t\mapsto Y^{\gamma\delta}_t(\omega)$ is right-continuous at every $t\in[0,T)$ in the $\varepsilon$-$\eta$ sense.

4. (Stopped martingale and covariation identities.) Let $\tau$ be a stopping time of $(\mathcal{F}^{\mathrm{sys}}_t)_{t\in[0,T]}$, and for $t\in[0,T]$ let $\min(t,\tau)$ denote the function $\omega\mapsto\min(t,\tau(\omega))$. For all $0\le r\le t\le T$, the functions

are random variables, bounded in absolute value by $K_M$, $K_M^2$, and $2(l-1)B\,(t-r)$ respectively. Moreover, for all $0\le r\le t\le T$ and every event $D\in\mathcal{F}^{\mathrm{sys}}_r$,

and