Proof of Compensated Counters of the Controlled N-Agent Dynamics are Square-Integrable Martingales

Fix a solution; let $n_c=Nl(l-1)+N\tilde{l}$ be the number of clock labels and $\bar{\beta}=\max(B,\tilde{B})$, so every consumed clock time is $\bar{\beta}$-Lipschitz in $t$ by condition 2 of Solution of the Controlled N-Agent Dynamics. Adaptedness of $N^a$ and $\mathcal{T}^a$, square-integrability of $N^a_t$, and measurability of the time-$r$ configuration are provided by parts (iii) and (iv) of Existence, Uniqueness, and Regularity for the Controlled N-Agent Dynamics; in particular each $M^a_t=N^a_t-\mathcal{T}^a_t$ is adapted and square-integrable, and $M^a_0=N^a_0-\mathcal{T}^a_0=0$ since $\mathcal{T}^a_0=0$ (an integral over the null set $\{0\}$, by the integral toolkit) and hence $N^a_0=Y^a_0=0$ (clock paths are counting paths).

Step 1: the interval estimates. Fix $r\in[0,T)$ and $0<\delta\le T-r$, write $\Delta N^a=N^a_{r+\delta}-N^a_r$, $\Delta\mathcal{T}^a=\mathcal{T}^a_{r+\delta}-\mathcal{T}^a_r\in[0,\bar{\beta}\delta]$, $\mu=\bar{\beta}\delta$, and let $\hat{Y}^a$ be the residual clocks of the fresh-start property at time $r$: they are rate-$1$ Poisson processes with counting paths, mutually independent and jointly independent of $\mathcal{F}^{\mathrm{sys}}_r$. On the regular event, the increments of $N^a$ over $(r,r+\delta]$ are increments of $Y^a$ over $(\mathcal{T}^a_r,\mathcal{T}^a_{r+\delta}]\subseteq(\mathcal{T}^a_r,\mathcal{T}^a_r+\mu]$, so pathwise

$$0\le\Delta N^a\le\hat{Y}^a_{\mu}.$$

Let $R_b=\{\hat{Y}^b_{\mu}\ge1\}$, so $P(R_b)=1-e^{-\mu}\le\mu$, and let $R=\bigcup_{b\neq a}R_b$. Let $c^a$ be the frozen consumption: the integral over $(r,r+\delta]$ of the rate obtained by freezing the time-$r$ states and observation record. Explicitly, writing $\alpha^{\mathrm{fr}}(s)=h_{K_r}(s,\tau_1,\dots,\tau_{K_r},\upsilon_1,\dots,\upsilon_{K_r})$ for the control the policy produces from the time-$r$ record, set

$$\lambda^a_{\mathrm{frozen}}(s)=\mathbf{1}_{\{\sigma^i_r=\sigma\}}\,\beta\big(\sigma,\gamma,\Sigma_r,\alpha^{\mathrm{fr}}(s)\big)\quad\text{for }a=(i,\sigma\gamma),\qquad \lambda^a_{\mathrm{frozen}}(s)=\tilde{\beta}\big(\sigma^i_r,\upsilon,\Sigma_r\big)\quad\text{for }a=(i,\upsilon),$$

so that $c^a=\int_{(r,r+\delta]}\lambda^a_{\mathrm{frozen}}(s)\,ds\in[0,\mu]$. It is $\mathcal{F}^{\mathrm{sys}}_r$-measurable: by part (iv) of Existence, Uniqueness, and Regularity for the Controlled N-Agent Dynamics the time-$r$ states are $\mathcal{F}^{\mathrm{sys}}_r$-measurable and the observation-event count, event times, and channels up to $r$ are $\mathcal{G}_r$-measurable with $\mathcal{G}_r\subseteq\mathcal{F}^{\mathrm{sys}}_r$; the frozen rate path is a measurable function of these data by Sequentially Continuous Functions of Measurable Euclidean Maps are Measurable and Observation-Driven Control Policy, and the integral is measurable by Tonelli.

We claim

$$\{\Delta N^a\ge1\}\,\triangle\,\{\hat{Y}^a_{c^a}\ge1\}\subseteq R\cap\{\hat{Y}^a_{\mu}\ge1\}.$$

Indeed, suppose no clock $b\neq a$ rings in $(r,r+\delta]$ (that is, off $R$, since a ring of $b$ in $(r,r+\delta]$ is a jump of $Y^b$ within consumed budget at most $\mu$, hence forces $\hat{Y}^b_{\mu}\ge1$). Then up to the first ring of $a$ in the interval (or up to $r+\delta$ if there is none), no counter jumps, so the states and record stay at their time-$r$ values and the consumed time of $a$ follows the frozen schedule; consequently $a$ rings in $(r,r+\delta]$ if and only if the frozen consumption reaches the next level of $Y^a$ within the interval, that is, if and only if $\hat{Y}^a_{c^a}\ge1$. Thus on either difference set some clock $b\neq a$ must ring in the interval, putting the outcome in $R$; and on both difference sets $\hat{Y}^a_{\mu}\ge1$ (in the first case because $a$ actually rings within budget $\mu$, in the second case because $\hat{Y}^a_{c^a}\ge1$ and $c^a\le\mu$). By pairwise independence of the residual clocks, $P(R\cap\{\hat{Y}^a_{\mu}\ge1\})\le\sum_{b\neq a}P(R_b)P(R_a)\le n_c\mu^2$.

For any event $D\in\mathcal{F}^{\mathrm{sys}}_r$, joint measurability of $(c,\hat{\omega})\mapsto\hat{Y}^a_c$ (right-continuity and grid limits) together with independence of $\hat{Y}^a$ from $\mathcal{F}^{\mathrm{sys}}_r$ and Tonelli on the product of the two laws gives

$$\mathbb{E}\big[\mathbf{1}_D\,\mathbf{1}_{\{\hat{Y}^a_{c^a}\ge1\}}\big]=\mathbb{E}\big[\mathbf{1}_D\,(1-e^{-c^a})\big],$$

and, for $0\le c\le\mu$, $|1-e^{-c}-c|\le c^2/2\le\mu^2$: indeed $1-e^{-c}-c=-\int_{[0,c]}(1-e^{-s})\,ds$, while $0\le1-e^{-s}\le s$ for every $s\ge0$ by the exponential and its properties, so that integral is at most $\int_{[0,c]}s\,ds=c^2/2$. Moreover, for integers $0\le x\le y$ one has $(x-1)^+=x-\mathbf{1}_{\{x\ge1\}}\le y-\mathbf{1}_{\{y\ge1\}}$, so $\mathbb{E}[(\Delta N^a-1)^+]\le\mathbb{E}[\hat{Y}^a_\mu]-P(\hat{Y}^a_\mu\ge1)=\mu-(1-e^{-\mu})\le\mu^2$, by Moments of the Poisson Distribution. Finally, on the complement of $R\cup R_a$ no event occurs in $(r,r+\delta]$, so $\Delta\mathcal{T}^a=c^a$ there, while always $|\Delta\mathcal{T}^a-c^a|\le2\mu$; hence $|\mathbb{E}[(\Delta\mathcal{T}^a-c^a)\mathbf{1}_D]|\le2\mu\,P(R\cup R_a)\le2(n_c+1)\mu^2$. To assemble these, write $\Delta N^a=\mathbf{1}_{\{\Delta N^a\ge1\}}+(\Delta N^a-1)^+$ and estimate $\mathbb{E}[\Delta N^a\mathbf{1}_D]$ in four steps: the symmetric-difference bound replaces $\mathbf{1}_{\{\Delta N^a\ge1\}}$ by $\mathbf{1}_{\{\hat{Y}^a_{c^a}\ge1\}}$ at a cost of at most $n_c\mu^2$; the displayed identity and the exponential estimate replace the expectation of the latter by $\mathbb{E}[\mathbf{1}_Dc^a]$ at a further cost of at most $\mu^2$; the term $(\Delta N^a-1)^+$ contributes at most $\mu^2$; and $\mathbb{E}[\Delta\mathcal{T}^a\mathbf{1}_D]$ differs from $\mathbb{E}[c^a\mathbf{1}_D]$ by at most $2(n_c+1)\mu^2$. The terms $\mathbb{E}[\mathbf{1}_Dc^a]$ cancel on subtracting, and adding the four errors gives $(3n_c+4)\mu^2\le4(n_c+1)\mu^2$; hence

$$\big|\mathbb{E}\big[(\Delta N^a-\Delta\mathcal{T}^a)\mathbf{1}_D\big]\big|\le\mu^2+n_c\mu^2+\mu^2+2(n_c+1)\mu^2\ \le\ C_1\delta^2,\qquad C_1=4(n_c+1)\bar{\beta}^2.\tag{E1}$$

Two second-order estimates follow similarly. Since $(\Delta N^a)^2=\Delta N^a+\Delta N^a(\Delta N^a-1)\le\Delta N^a+\hat{Y}^a_\mu(\hat{Y}^a_\mu-1)$ and $\mathbb{E}[\hat{Y}^a_\mu(\hat{Y}^a_\mu-1)]=\mu^2$ by Moments of the Poisson Distribution, while $|\mathbb{E}[\Delta N^a\Delta\mathcal{T}^a\mathbf{1}_D]|\le\bar{\beta}\delta\,\mathbb{E}[\hat{Y}^a_\mu]=\mu^2$ and $(\Delta\mathcal{T}^a)^2\le\mu^2$, expanding $(\Delta M^a)^2=(\Delta N^a-\Delta\mathcal{T}^a)^2$ and using (E1) gives

$$\big|\mathbb{E}\big[\big((\Delta M^a)^2-\Delta\mathcal{T}^a\big)\mathbf{1}_D\big]\big|\le C_2\delta^2,\qquad C_2=C_1+4\bar{\beta}^2.\tag{E2}$$

For $b\neq a$: $\mathbb{E}[\Delta N^a\Delta N^b\mathbf{1}_D]\le\mathbb{E}[\hat{Y}^a_\mu\hat{Y}^b_\mu]=\mu^2$ by independence of the two residual clocks, and the mixed and quadratic $\Delta\mathcal{T}$-terms are bounded by $\mu^2$ as before, so

$$\big|\mathbb{E}\big[\Delta M^a\,\Delta M^b\,\mathbf{1}_D\big]\big|\le C_3\delta^2,\qquad C_3=4\bar{\beta}^2.\tag{E3}$$

Step 2: part (a). Fix $0\le r<t\le T$ and $D\in\mathcal{F}^{\mathrm{sys}}_r$. Partition $(r,t]$ into $n$ intervals of length $\delta=(t-r)/n$ with left endpoints $r_q$. Since $D\in\mathcal{F}^{\mathrm{sys}}_r\subseteq\mathcal{F}^{\mathrm{sys}}_{r_q}$ for every $q$ (filtration), (E1) applies on each interval and

$$\big|\mathbb{E}\big[(M^a_t-M^a_r)\mathbf{1}_D\big]\big|\le\sum_{q}C_1\delta^2=C_1(t-r)\delta\longrightarrow0\qquad(n\to\infty),$$

so $\mathbb{E}[M^a_t\mathbf{1}_D]=\mathbb{E}[M^a_r\mathbf{1}_D]$. This is the averaged martingale identity, which by Square-Integrable Martingale, Submartingale, and Supermartingale is equivalent to the martingale property for adapted square-integrable processes; hence $(M^a_t)_{t\in[0,T]}$ is a square-integrable martingale with $M^a_0=0$.

Step 3: part (b). Integrability of the products holds as noted in the statement, by part (iii) of Existence, Uniqueness, and Regularity for the Controlled N-Agent Dynamics and the Cauchy--Schwarz inequality. First observe that the averaged identity of Step 2 extends from indicators to arbitrary square-integrable $\mathcal{F}^{\mathrm{sys}}_{r_q}$-measurable multipliers $Z$: truncating $Z$ at level $n'$ and discretizing the truncation's values on a dyadic grid produces simple $\mathcal{F}^{\mathrm{sys}}_{r_q}$-measurable variables $Z_{n'}$ with $|Z_{n'}|\le|Z|+1$ and $Z_{n'}\to Z$ pointwise; $\mathbb{E}[(M^a_{t'}-M^a_{r_q})Z_{n'}]=0$ by linearity over the finitely many level sets, and $\mathbb{E}[(M^a_{t'}-M^a_{r_q})(Z-Z_{n'})]\to0$ by the Cauchy--Schwarz inequality together with dominated convergence applied to $|Z-Z_{n'}|^2\le(2|Z|+1)^2$; thus $\mathbb{E}[(M^a_{t'}-M^a_{r_q})Z]=0$ for $t'\ge r_q$. Now with the partition of Step 2 write, for clock labels $a,b$,

$$M^a_tM^b_t-M^a_rM^b_r=\sum_q\Big(\Delta M^a_q\,\Delta M^b_q+M^a_{r_q}\,\Delta M^b_q+M^b_{r_q}\,\Delta M^a_q\Big),$$

where $\Delta M^a_q$ is the increment over the $q$-th interval. Multiplying by $\mathbf{1}_D$ and taking expectations, the cross terms vanish by the extended averaged identity applied with $Z=M^a_{r_q}\mathbf{1}_D$ and $Z=M^b_{r_q}\mathbf{1}_D$ (both square-integrable and $\mathcal{F}^{\mathrm{sys}}_{r_q}$-measurable). If $a\neq b$, (E3) gives

$$\big|\mathbb{E}\big[(M^a_tM^b_t-M^a_rM^b_r)\mathbf{1}_D\big]\big|\le\sum_qC_3\delta^2=C_3(t-r)\delta\longrightarrow0,$$

which is the case $a\neq b$ of the display in (b). If $a=b$, (E2) gives

$$\Big|\mathbb{E}\big[(M^a_tM^a_t-M^a_rM^a_r)\mathbf{1}_D\big]-\sum_q\mathbb{E}\big[\Delta\mathcal{T}^a_q\mathbf{1}_D\big]\Big|\le C_2(t-r)\delta,$$

and $\sum_q\mathbb{E}[\Delta\mathcal{T}^a_q\mathbf{1}_D]=\mathbb{E}[(\mathcal{T}^a_t-\mathcal{T}^a_r)\mathbf{1}_D]$ exactly, for every $n$, by telescoping. Letting $n\to\infty$ yields

$$\mathbb{E}\big[M^a_tM^a_t\,\mathbf{1}_D\big]=\mathbb{E}\big[M^a_rM^a_r\,\mathbf{1}_D\big]+\mathbb{E}\big[\mathbf{1}_D(\mathcal{T}^a_t-\mathcal{T}^a_r)\big],$$

which is the case $a=b$. $\blacksquare$