Proof of Stopped Covariation Identities for the Martingale Part of the Empirical State Measure

Throughout, fix the solution and its derived objects as in the statement, write $\mathbf{1}=\mathbf{1}_{\Omega_0}$, and abbreviate the martingale decomposition theorem as the decomposition theorem and the existence theorem for the controlled $N$-agent dynamics as the existence theorem. Since $P(\Omega_0)=1$, expectations are unchanged when integrands are modified off $\Omega_0$; we use this silently. We record two remarks.

Bounded-integrability remark. A measurable real-valued function bounded in absolute value by a real $c\ge0$ is integrable on a finite measure space. On a compact interval with the restricted Lebesgue measure: its square has integral at most that of the constant $c^2$ by monotonicity of the nonnegative integral, the constant being integrable with finite integral by claim 3 of the integral toolkit (constants are continuous), so claim 4 of the toolkit with $g=1$ shows the absolute value has finite integral. On $(\Omega,\mathcal{F},P)$: a random variable $X$ with $|X|\le c$ has $\int_\Omega|X|\,dP\le c\,P(\Omega)=c<\infty$ by monotonicity, the nonnegative integral of the constant $c$ being $c\,P(\Omega)$ directly from the definition (a simple function); hence $X$ is integrable, and if $|X|\le c$ then $X^2$ is a random variable bounded by $c^2$, so $X$ is also square-integrable.

Additivity remark. Let $0\le r\le t\le T$ and let $u:[0,T]\to\mathbb{R}$ be measurable with respect to $\mathcal{B}_{[0,T]}$, the trace Borel $\sigma$-algebra, and the Borel $\sigma$-algebra of the real line, with $|u|\le K$ for a real $K\ge0$. Then

$$\int_{[0,t]}u(s)\,ds=\int_{[0,r]}u(s)\,ds+\int_{[r,t]}u(s)\,ds .$$

The cases $r=0$ and $r=t$ are trivial by the conventions $\int_{[0,0]}\cdot\,ds=0$ and $\int_{[t,t]}\cdot\,ds=0$, so let $0<r<t$. Restrictions of $u$ to compact subintervals are measurable (the preimage under a restriction is the intersection of the original preimage with the subinterval, and a trace of a trace is a trace) and bounded, hence integrable by the bounded-integrability remark. The zero extension to $\mathbb{R}$ of the restriction of $u$ to $[0,r]$ coincides with the zero extension of $s\mapsto u(s)\mathbf{1}_{[0,r]}(s)$ on $[0,t]$, where $\mathbf{1}_{[0,r]}$ is the indicator of $[0,r]$; so two applications of claim 2 of the integral toolkit give $\int_{[0,r]}u=\int_{[0,t]}u\,\mathbf{1}_{[0,r]}$, and likewise $\int_{[r,t]}u=\int_{[0,t]}u\,\mathbf{1}_{[r,t]}$. Since $\mathbf{1}_{[0,r]}+\mathbf{1}_{[r,t]}=1+\mathbf{1}_{\{r\}}$ on $[0,t]$, linearity gives $\int_{[0,r]}u+\int_{[r,t]}u=\int_{[0,t]}u+\int_{[0,t]}u\,\mathbf{1}_{\{r\}}$, and the last integral vanishes by claim 1 of the null-set integral lemma: the integrand vanishes off $\{r\}=[r,r]$, a null set by the interval-length property of Lebesgue measure.

Step 1 (product measurability and the two indefinite integrals). By parts (b) and (c) of the joint measurability lemma, the maps $(s,\omega)\mapsto\mathbf{1}\Sigma^\gamma_s$ and $(s,\omega)\mapsto\mathbf{1}\alpha^j_s$ are measurable with respect to the product $\sigma$-algebra $\mathcal{B}_{[0,T]}\otimes\mathcal{F}$. Fix a point $(\Sigma^*,\alpha^*)$ of $\Delta^l\times\mathcal{A}$ ($\mathcal{A}$ is nonempty) and define the modified map $(s,\omega)\mapsto(\Sigma^\circ_s,\alpha^\circ_s)$ equal to $(\Sigma_s,\alpha_s)$ on $[0,T]\times\Omega_0$ and to $(\Sigma^*,\alpha^*)$ off it; each of its components, e.g. $\Sigma^{\circ\gamma}_s=\mathbf{1}\Sigma^\gamma_s+(1-\mathbf{1})\Sigma^{*\gamma}$, is product-measurable by measurability of sequentially continuous functions of measurable maps (sums and products with the product-measurable $(s,\omega)\mapsto\mathbf{1}(\omega)$). Each of the functions $(\Sigma,\alpha)\mapsto b^\gamma(\Sigma,\alpha)$ and $(\Sigma,\alpha)\mapsto\Theta^{\gamma\delta}(\Sigma,\alpha)$ is a finite sum of products of coordinate maps and members of the transition-rate family — by the formulas of the aggregate state drift and of the aggregate fluctuation covariance — hence sequentially continuous on $\Delta^l\times\mathcal{A}$, by the joint continuity condition of the transition-rate family and the arithmetic of limits of real sequences. So the composition lemma applies to the componentwise product-measurable modified map and yields product-measurability of $(s,\omega)\mapsto b^\gamma(\Sigma^\circ_s,\alpha^\circ_s)$ and $(s,\omega)\mapsto\Theta^{\gamma\delta}(\Sigma^\circ_s,\alpha^\circ_s)$; multiplying by $\mathbf{1}$ — which changes nothing on $[0,T]\times\Omega_0$ and makes all the maps vanish off it — shows that

$$(s,\omega)\mapsto\mathbf{1}\,b^\gamma(\Sigma_s,\alpha_s)\qquad\text{and}\qquad(s,\omega)\mapsto\mathbf{1}\,\Theta^{\gamma\delta}(\Sigma_s,\alpha_s)$$

are measurable with respect to $\mathcal{B}_{[0,T]}\otimes\mathcal{F}$ and bounded in absolute value by $2(l-1)B$ everywhere (part (a) of the decomposition theorem, the bounds holding at every point of $\Omega$).

Both maps are therefore progressively measurable with respect to the constant filtration $(\mathcal{C}_t)_{t\in[0,T]}$, $\mathcal{C}_t=\mathcal{F}$ — a filtration with time index restricted to $[0,T]$. Indeed the family of subsets of $[0,T]\times\Omega$ whose intersection with $[0,t]\times\Omega$ lies in $\mathcal{B}_{[0,t]}\otimes\mathcal{F}$ is a $\sigma$-algebra containing every measurable rectangle $A\times E$ (as $A\cap[0,t]\in\mathcal{B}_{[0,t]}$, a trace of a trace being a trace), hence contains $\mathcal{B}_{[0,T]}\otimes\mathcal{F}$; so preimages under the restriction to $[0,t]\times\Omega$ are measurable. By claim 4 of the progressive measurability toolkit, applied with the constant filtration and the bound $2(l-1)B$: for every $\omega\in\Omega$ and $t\in[0,T]$ the integrals

$$\int_{[0,t]}\mathbf{1}\,b^\gamma(\Sigma_s,\alpha_s)\,ds\qquad\text{and}\qquad \mathcal{Q}^{\gamma\delta}_t=\int_{[0,t]}\mathbf{1}\,\Theta^{\gamma\delta}(\Sigma_s,\alpha_s)\,ds$$

are defined, and each of the two indefinite-integral families has increments bounded by $2(l-1)B\,(t-r)$ over $0\le r\le t\le T$ at every $\omega$ and has every path continuous on $[0,T]$.

Step 2 (claim 1). At every $(s,\omega)\in[0,T]\times\Omega$,

$$\mathbf{1}\,\Theta^{\gamma\delta}(\Sigma_s,\alpha_s)=\sum_{(\sigma,\gamma'):\,\sigma\neq\gamma'}w^{\gamma\delta}_{\sigma\gamma'}\,\frac{1}{N}\sum_{i=1}^{N}\mathbf{1}\,\eta^{i,\sigma}_s\,\beta(\sigma,\gamma',\Sigma_s,\alpha_s),\qquad w^{\gamma\delta}_{\sigma\gamma'}=\big(\mathbf{1}_{\{\gamma'=\gamma\}}-\mathbf{1}_{\{\sigma=\gamma\}}\big)\big(\mathbf{1}_{\{\gamma'=\delta\}}-\mathbf{1}_{\{\sigma=\delta\}}\big).$$

Off $\Omega_0$ both sides vanish. At any $\omega$, $\frac{1}{N}\sum_i\eta^{i,\sigma}_s=\Sigma^\sigma_s$ by the derived notation of the solution definition, so it suffices to check, for every $(\Sigma,\alpha)\in\Delta^l\times\mathcal{A}$,

$$\sum_{(\sigma,\gamma'):\,\sigma\neq\gamma'}w^{\gamma\delta}_{\sigma\gamma'}\,\Sigma^\sigma\,\beta(\sigma,\gamma',\Sigma,\alpha)=\Theta^{\gamma\delta}(\Sigma,\alpha).$$

For $\gamma=\delta$: $w^{\gamma\gamma}_{\sigma\gamma'}=(\mathbf{1}_{\{\gamma'=\gamma\}}-\mathbf{1}_{\{\sigma=\gamma\}})^2$ equals $1$ exactly when exactly one of $\gamma'=\gamma$, $\sigma=\gamma$ holds (both cannot hold, since $\sigma\neq\gamma'$) and $0$ otherwise, so the left side is $\sum_{\sigma\neq\gamma}\Sigma^\sigma\beta(\sigma,\gamma,\Sigma,\alpha)+\sum_{\gamma'\neq\gamma}\Sigma^\gamma\beta(\gamma,\gamma',\Sigma,\alpha)$, which is the diagonal entry $\Theta^{\gamma\gamma}(\Sigma,\alpha)$ of the aggregate fluctuation covariance after relabeling the second summation index. For $\gamma\neq\delta$: $w^{\gamma\delta}_{\sigma\gamma'}\neq0$ requires ($\gamma'=\gamma$ or $\sigma=\gamma$) and ($\gamma'=\delta$ or $\sigma=\delta$); since $\gamma\neq\delta$ and $\sigma\neq\gamma'$, the only such pairs are $(\sigma,\gamma')=(\gamma,\delta)$, where $w^{\gamma\delta}_{\gamma\delta}=(0-1)(1-0)=-1$, and $(\sigma,\gamma')=(\delta,\gamma)$, where $w^{\gamma\delta}_{\delta\gamma}=(1-0)(0-1)=-1$; the left side is $-\Sigma^\gamma\beta(\gamma,\delta,\Sigma,\alpha)-\Sigma^\delta\beta(\delta,\gamma,\Sigma,\alpha)=\Theta^{\gamma\delta}(\Sigma,\alpha)$, the off-diagonal formula.

For fixed $\omega$, the sections in $s$ of the finitely many maps $(s,\omega)\mapsto\mathbf{1}\eta^{i,\sigma}_s\beta(\sigma,\gamma',\Sigma_s,\alpha_s)$ are measurable on $[0,t]$ with values in $[0,B]$, and the integral over $[0,t]$ of each section is the consumed clock time $\mathcal{T}^{i,\sigma\gamma'}_t(\omega)$: condition 2 of the solution definition defines the consumed clock times as exactly these Lebesgue integrals of sections (so the sections' measurability and the existence of the integrals at every $\omega$ are part of that definition), and the integrands take values in $[0,B]$ by part (vii)(b) of the existence theorem. Integrating the pointwise identity above over $[0,t]$ and using linearity (finitely many bounded measurable integrands, integrable by the bounded-integrability remark) yields, at every point of $\Omega$ and for every $t\in(0,T]$,

$$\mathcal{Q}^{\gamma\delta}_t=\sum_{(\sigma,\gamma'):\,\sigma\neq\gamma'}w^{\gamma\delta}_{\sigma\gamma'}\,\frac{1}{N}\sum_{i=1}^{N}\mathcal{T}^{i,\sigma\gamma'}_t ;$$

for $t=0$ both sides vanish, the left by the convention and the right because $0\le\mathcal{T}^{i,\sigma\gamma'}_0\le0$ by part (vii)(b) of the existence theorem. This is the asserted representation.

Each $\mathcal{T}^{i,\sigma\gamma'}_t$ is $\mathcal{F}^{\mathrm{sys}}_t$-measurable (part (iv) of the existence theorem: the consumed clock times are adapted), and a finite linear combination of measurable real-valued functions is measurable (the composition lemma, a linear function of finitely many real variables being sequentially continuous); so $\mathcal{Q}^{\gamma\delta}_t$ is $\mathcal{F}^{\mathrm{sys}}_t$-measurable for every $t$, that is, $\mathcal{Q}^{\gamma\delta}$ is adapted. By Step 1, $|\mathcal{Q}^{\gamma\delta}_t-\mathcal{Q}^{\gamma\delta}_r|\le2(l-1)B(t-r)$ at every $\omega$ and every path of $\mathcal{Q}^{\gamma\delta}$ is continuous; with $\mathcal{Q}^{\gamma\delta}_0=0$ this gives $|\mathcal{Q}^{\gamma\delta}_t|\le2(l-1)BT$ everywhere. Every path is right-continuous in the sequential sense: for a sequence $(s_j)$ in $[t,T]$ converging to $t$, $|\mathcal{Q}^{\gamma\delta}_{s_j}-\mathcal{Q}^{\gamma\delta}_t|\le2(l-1)B(s_j-t)\to0$. Hence the adapted family $\mathcal{Q}^{\gamma\delta}$ is progressively measurable by claim 2 of the progressive measurability toolkit. This proves claim 1.

Step 3 (claim 2). At every point of $\Omega$ and for every $u\in[0,T]$, $\Sigma_u\in\Delta^l$ by part (vii)(a) of the existence theorem, and the components of a point of the probability simplex are nonnegative with sum $1$, hence lie in $[0,1]$. So $|\Sigma^\gamma_t-\Sigma^\gamma_0|\le1$ everywhere, and $|\int_{[0,t]}\mathbf{1}b^\gamma(\Sigma_s,\alpha_s)ds|\le2(l-1)Bt$ by the increment bound of Step 1 (against $r=0$, the integral at $0$ vanishing), giving $|M^\gamma_t|\le1+2(l-1)BT=K_M$ at every point of $\Omega$.

Right-continuity. Fix $\omega\in\Omega_0$ and $t\in[0,T)$. By condition 1 of the solution definition, for each $i$ the path $u\mapsto\sigma^i_u(\omega)$ is constant on finitely many consecutive intervals covering $[0,T]$, each of the form $[a,c)$ except the last, of the form $[a,T]$. If $t$ lies in a piece $[a,c)$, put $\eta_i=(c-t)/2>0$, so that the path is constant on $[t,t+\eta_i]\subseteq[a,c)$; if $t$ lies in the final piece $[a,T]$, put $\eta_i=T-t>0$, the path being constant on $[t,T]$. Let $\eta=\min_i\eta_i>0$. Then for every $s$ with $t\le s\le\min(t+\eta,T)$ and every $i$: $\sigma^i_s(\omega)=\sigma^i_t(\omega)$, hence $\eta^{i,\gamma}_s(\omega)=\eta^{i,\gamma}_t(\omega)$ for all $\gamma$ and $\Sigma^\gamma_s(\omega)=\Sigma^\gamma_t(\omega)$. In particular the path of $\Sigma^\gamma$ is right-continuous at $t$ in the $\varepsilon$-$\eta$ sense, the increment being $0$. For $M^\gamma$ and such $s$,

$$|M^\gamma_s(\omega)-M^\gamma_t(\omega)|=\Big|\int_{[0,s]}\mathbf{1}\,b^\gamma(\Sigma_u,\alpha_u)\,du-\int_{[0,t]}\mathbf{1}\,b^\gamma(\Sigma_u,\alpha_u)\,du\Big|\le2(l-1)B\,(s-t)$$

by the increment bound of Step 1. Given $\varepsilon>0$, put $\eta'=\min\big(\eta,\ \varepsilon/(2(l-1)B+1)\big)>0$; then $|M^\gamma_s(\omega)-M^\gamma_t(\omega)|\le\varepsilon$ whenever $t\le s\le\min(t+\eta',T)$, which is right-continuity at $t$ in the $\varepsilon$-$\eta$ sense.

Progressive measurability. The set $\Omega\setminus\Omega_0$ is an event of probability zero, and by the solution definition the system filtration contains every event of $\mathcal{F}$ of probability zero; hence $\Omega\setminus\Omega_0\in\mathcal{F}^{\mathrm{sys}}_t$ and, by closure under complements, $\Omega_0\in\mathcal{F}^{\mathrm{sys}}_t$ for every $t$. Each $\Sigma^\gamma_t$ is $\mathcal{F}^{\mathrm{sys}}_t$-measurable (part (iv) of the existence theorem), and each $M^\gamma_t$ is $\mathcal{F}^{\mathrm{sys}}_t$-measurable (part (b) of the decomposition theorem: a square-integrable martingale is by definition adapted); products of measurable real-valued functions are measurable (the composition lemma), so the families $\mathbf{1}\Sigma^\gamma$ and $\mathbf{1}M^\gamma$ are adapted. Every path of each is right-continuous in the sequential sense: off $\Omega_0$ the paths are identically $0$; on $\Omega_0$, at $t=T$ every sequence in $[T,T]$ is constant, and at $t\in[0,T)$, for a sequence $(s_j)$ in $[t,T]$ converging to $t$ and any $\varepsilon>0$, eventually $t\le s_j\le\min(t+\eta',T)$ with the $\eta'$ of the $\varepsilon$-$\eta$ property just proved, so the increments are eventually at most $\varepsilon$ — that is, the path values converge. By claim 2 of the progressive measurability toolkit, $\mathbf{1}\Sigma^\gamma$ and $\mathbf{1}M^\gamma$ are progressively measurable. This proves claim 2.

Step 4 (claim 3). Each $Y^{\gamma\delta}_t$ is $\mathcal{F}^{\mathrm{sys}}_t$-measurable (products and linear combinations of the $\mathcal{F}^{\mathrm{sys}}_t$-measurable functions $M^\gamma_t$, $M^\delta_t$, $\mathcal{Q}^{\gamma\delta}_t$, by the composition lemma), and at every point of $\Omega$, using claims 1 and 2 and $N\ge1$,

$$|Y^{\gamma\delta}_t|\le K_M^2+\tfrac{1}{N}\,2(l-1)BT\le K_M^2+2(l-1)BT=:K_Y .$$

By the bounded-integrability remark, each $Y^{\gamma\delta}_t$ is square-integrable. Martingale property: fix $0\le r\le t\le T$ and $D\in\mathcal{F}^{\mathrm{sys}}_r$. At every $\omega$, the additivity remark applied to the section $s\mapsto\mathbf{1}(\omega)\Theta^{\gamma\delta}(\Sigma_s(\omega),\alpha_s(\omega))$ — measurable on $[0,T]$ at every $\omega$ and bounded by $2(l-1)B$, by part (a) of the decomposition theorem — gives

$$\mathcal{Q}^{\gamma\delta}_t-\mathcal{Q}^{\gamma\delta}_r=\int_{[r,t]}\mathbf{1}\,\Theta^{\gamma\delta}(\Sigma_s,\alpha_s)\,ds .$$

All products appearing below are integrable: they are bounded random variables (claims 1, 2, and the bounded-integrability remark). By linearity of the integral and part (c) of the decomposition theorem,

$$\mathbb{E}\big[Y^{\gamma\delta}_t\mathbf{1}_D\big]-\mathbb{E}\big[Y^{\gamma\delta}_r\mathbf{1}_D\big]=\frac{1}{N}\,\mathbb{E}\Big[\mathbf{1}_D\int_{[r,t]}\mathbf{1}\Theta^{\gamma\delta}(\Sigma_s,\alpha_s)\,ds\Big]-\frac{1}{N}\,\mathbb{E}\big[\mathbf{1}_D\,\big(\mathcal{Q}^{\gamma\delta}_t-\mathcal{Q}^{\gamma\delta}_r\big)\big]=0 .$$

The family $Y^{\gamma\delta}$ is adapted with square-integrable entries and satisfies the averaged identity for all $0\le r\le t\le T$ and $D\in\mathcal{F}^{\mathrm{sys}}_r$, hence is a square-integrable martingale by the equivalence recorded in the definition of a square-integrable martingale (time index restricted to $[0,T]$).

Right-continuity on $\Omega_0$: fix $\omega\in\Omega_0$, $t\in[0,T)$, and $\varepsilon>0$. For $t\le s\le T$,

$$|Y^{\gamma\delta}_s-Y^{\gamma\delta}_t|\le|M^\gamma_s|\,|M^\delta_s-M^\delta_t|+|M^\delta_t|\,|M^\gamma_s-M^\gamma_t|+\tfrac{1}{N}|\mathcal{Q}^{\gamma\delta}_s-\mathcal{Q}^{\gamma\delta}_t|\le K_M\big(|M^\delta_s-M^\delta_t|+|M^\gamma_s-M^\gamma_t|\big)+2(l-1)B\,(s-t).$$

By claim 2, choose $\eta_1>0$ so that both martingale increments are at most $\varepsilon/(4K_M+2)$ for $t\le s\le\min(t+\eta_1,T)$, and put $\eta_2=\varepsilon/(4(l-1)B+2)$ and $\eta=\min(\eta_1,\eta_2)$; then for $t\le s\le\min(t+\eta,T)$ the right side is at most $2K_M\,\varepsilon/(4K_M+2)+2(l-1)B\,\varepsilon/(4(l-1)B+2)\le\varepsilon/2+\varepsilon/2=\varepsilon$. This proves claim 3.

Step 5 (claim 4). Let $\tau$ be a stopping time of $(\mathcal{F}^{\mathrm{sys}}_t)_{t\in[0,T]}$ and fix $0\le r\le t\le T$; the functions $\min(t,\tau)$ and $\min(r,\tau)$ are stopping times by claim 1 of the stopping-time toolkit (constants are stopping times, and the pointwise minimum of two stopping times is one).

Random variables and bounds. The filtration $(\mathcal{F}^{\mathrm{sys}}_t)_{t\in[0,T]}$ (a filtration by the solution definition), the event $\Omega_0$ (with $P(\Omega_0)=1$), and each of the processes $M^\gamma$ — a square-integrable martingale by part (b) of the decomposition theorem, bounded everywhere by $K_M$ and with paths right-continuous at every $t\in[0,T)$ on $\Omega_0$ in the $\varepsilon$-$\eta$ sense, by claim 2 — and $Y^{\gamma\delta}$ — the same with bound $K_Y$, by claim 3 — satisfy the hypotheses of the optional stopping theorem. By part (a) of that theorem applied with the stopping time $\min(t,\tau)$, the functions $\mathbf{1}M^\gamma_{\min(t,\tau)}$ and $\mathbf{1}Y^{\gamma\delta}_{\min(t,\tau)}$ are random variables bounded by $K_M$ and $K_Y$. Since $\mathcal{Q}^{\gamma\delta}$ is progressively measurable (claim 1), the sampled function $\mathcal{Q}^{\gamma\delta}_{\min(t,\tau)}$ is a random variable by claim 4(ii) of the stopping-time toolkit. Sampling is pointwise evaluation, so at every $\omega$

$$Y^{\gamma\delta}_{\min(t,\tau)}=M^\gamma_{\min(t,\tau)}M^\delta_{\min(t,\tau)}-\tfrac{1}{N}\,\mathcal{Q}^{\gamma\delta}_{\min(t,\tau)},$$

whence $\mathbf{1}\,M^\gamma_{\min(t,\tau)}M^\delta_{\min(t,\tau)}=\mathbf{1}Y^{\gamma\delta}_{\min(t,\tau)}+\tfrac1N\mathbf{1}\mathcal{Q}^{\gamma\delta}_{\min(t,\tau)}$ is a random variable; it is bounded by $K_M^2$, the product of two factors bounded by $K_M$ (claim 2).

For the integral: at every $\omega$, the section $s\mapsto\mathbf{1}(\omega)\,\Theta^{\gamma\delta}(\Sigma_s(\omega),\alpha_s(\omega))$ is measurable on $[0,T]$ and bounded by $2(l-1)B$ (part (a) of the decomposition theorem; off $\Omega_0$ it vanishes identically), so claim 2 of the stopped-time-integral lemma gives, for $u\in\{r,t\}$ and at every $\omega$,

$$\mathcal{Q}^{\gamma\delta}_{\min(u,\tau(\omega))}(\omega)=\int_{[0,u]}\mathbf{1}_{\{s<\tau\}}(\omega)\,\mathbf{1}(\omega)\,\Theta^{\gamma\delta}(\Sigma_s(\omega),\alpha_s(\omega))\,ds,$$

and the additivity remark (applied at each $\omega$ to the measurable bounded section $s\mapsto\mathbf{1}_{\{s<\tau\}}(\omega)\mathbf{1}(\omega)\Theta^{\gamma\delta}(\Sigma_s(\omega),\alpha_s(\omega))$, a product of measurable functions by claim 1 of the stopped-time-integral lemma and the composition lemma) yields, at every $\omega$,

$$\int_{[r,t]}\mathbf{1}_{\{s<\tau\}}\,\mathbf{1}\,\Theta^{\gamma\delta}(\Sigma_s,\alpha_s)\,ds=\mathcal{Q}^{\gamma\delta}_{\min(t,\tau)}-\mathcal{Q}^{\gamma\delta}_{\min(r,\tau)} .$$

The right side is a random variable, with absolute value at most $2(l-1)B\,\big(\min(t,\tau)-\min(r,\tau)\big)\le2(l-1)B\,(t-r)$ by the increment bound of claim 1. This proves the measurability and boundedness assertions of claim 4.

Identities. Let $D\in\mathcal{F}^{\mathrm{sys}}_r$. Part (c) of the optional stopping theorem applied to $M^\gamma$, with the pair of times $r\le t$ and the event $D$, states

$$\mathbb{E}\big[\mathbf{1}\,M^\gamma_{\min(t,\tau)}\,\mathbf{1}_D\big]=\mathbb{E}\big[\mathbf{1}\,M^\gamma_{\min(r,\tau)}\,\mathbf{1}_D\big],$$

the stopped family of the sampling definition having $M^{\gamma,\tau}_u=M^\gamma_{\min(u,\tau)}$; this is the first identity. Applied to $Y^{\gamma\delta}$ it gives $\mathbb{E}[\mathbf{1}Y^{\gamma\delta}_{\min(t,\tau)}\mathbf{1}_D]=\mathbb{E}[\mathbf{1}Y^{\gamma\delta}_{\min(r,\tau)}\mathbf{1}_D]$. Substituting the pointwise decomposition of $Y^{\gamma\delta}_{\min(u,\tau)}$ displayed above for $u=t$ and $u=r$, and rearranging by linearity (all terms bounded random variables, integrable by the bounded-integrability remark),

$$\mathbb{E}\big[\mathbf{1}\,M^\gamma_{\min(t,\tau)}M^\delta_{\min(t,\tau)}\,\mathbf{1}_D\big]-\mathbb{E}\big[\mathbf{1}\,M^\gamma_{\min(r,\tau)}M^\delta_{\min(r,\tau)}\,\mathbf{1}_D\big]=\frac{1}{N}\,\mathbb{E}\big[\mathbf{1}\,\big(\mathcal{Q}^{\gamma\delta}_{\min(t,\tau)}-\mathcal{Q}^{\gamma\delta}_{\min(r,\tau)}\big)\,\mathbf{1}_D\big].$$

At every $\omega$, $\mathbf{1}(\omega)\big(\mathcal{Q}^{\gamma\delta}_{\min(t,\tau)}(\omega)-\mathcal{Q}^{\gamma\delta}_{\min(r,\tau)}(\omega)\big)$ equals $\int_{[r,t]}\mathbf{1}_{\{s<\tau\}}\mathbf{1}\Theta^{\gamma\delta}(\Sigma_s,\alpha_s)\,ds$: off $\Omega_0$ both sides vanish, the integrand carrying the factor $\mathbf{1}(\omega)=0$, while on $\Omega_0$ this is the pathwise display above. Substituting gives the second identity. $\blacksquare$