Proof of Forward Equation for a Finite-State Jump System Driven by Poisson Clocks with the Fresh-Start Property

Throughout, $\|F\|=\max_{x\in E}|F(x)|$, and linearity and monotonicity of the integral are used freely for bounded measurable integrands on compact intervals (such integrands are integrable, the interval having finite measure by claim 1 of Restricted Lebesgue Measure and Integral Toolkit on a Compact Interval), including the bound $\bigl|\int f\bigr|\le\int|f|$ of claim 2 of that theorem. Natural numbers are as in Natural Numbers.

Preliminaries. (P1) Joint measurability. Let $H:[0,T]\times E\to\mathbb{R}$ be bounded with $u\mapsto H(u,x)$ measurable on $[0,T]$ for each $x$. Then $(u,\omega)\mapsto\mathbf{1}_{\Omega_0}(\omega)H(u,X_u(\omega))=\sum_{x\in E}H(u,x)\mathbf{1}_{\Omega_0}(\omega)\mathbf{1}\{X_u(\omega)=x\}$ is $\mathcal{B}_{[0,T]}\otimes\mathcal{F}$-measurable, by (D3) and Arithmetic, Absolute Values, and Pointwise Limits of Measurable Real-Valued Functions, since $(u,\omega)\mapsto H(u,x)$ is product-measurable (its preimages are rectangles $S\times\Omega$ with $S\in\mathcal{B}_{[0,T]}$). For $0\le s\le t\le T$ the integral over $[s,t]$ of a bounded $\mathcal{B}_{[0,T]}$-measurable $f$ equals the integral over $[0,T]$ of $\mathbf{1}_{[s,t]}f$, both being the integral over $\mathbb{R}$ of the same zero extension (claim 2 of Restricted Lebesgue Measure and Integral Toolkit on a Compact Interval), and $(u,\omega)\mapsto\mathbf{1}_{[s,t]}(u)$ is product-measurable by the rectangle argument. Hence, by the Tonelli theorem on $[0,T]\times\Omega$ (applied to the positive and negative parts of $\mathbf{1}_{[s,t]}(u)\mathbf{1}_{\Omega_0}H(u,X_u)$), the map $\omega\mapsto\int_{[s,t]}\mathbf{1}_{\Omega_0}H(u,X_u)\,du$ is defined everywhere and $\mathcal{F}$-measurable, and for every $D\in\mathcal{F}$,

$$\mathbb{E}\Bigl[\mathbf{1}_D\int_{[s,t]}\mathbf{1}_{\Omega_0}H(u,X_u)\,du\Bigr]=\int_{[s,t]}\mathbb{E}\bigl[\mathbf{1}_D\mathbf{1}_{\Omega_0}H(u,X_u)\bigr]\,du,$$

the inner expectation being a measurable function of $u$. (P2) Adjacent intervals. For a bounded $\mathcal{B}_{[0,T]}$-measurable $f$ and $0\le s\le t\le T$ one has $\int_{[0,t]}f\,du-\int_{[0,s]}f\,du=\int_{[s,t]}f\,du$ and $\bigl|\int_{[s,t]}f\,du\bigr|\le(t-s)\sup|f|$: by claim 2 of Restricted Lebesgue Measure and Integral Toolkit on a Compact Interval each integral equals the integral over $\mathbb{R}$ of the corresponding zero extension, the zero extensions from $[0,t]$ and $[0,s]$ differ by the zero extension from $(s,t]$, whose integral equals that of the zero extension from $[s,t]$ because the two agree off the singleton $\{s\}$, which has Lebesgue measure $0$ (claim 6 of Restricted Lebesgue Measure and Integral Toolkit on a Compact Interval, applied on $[s,t]$ with $D=(s,t]$, for $s<t$); and $[s,t]$ has measure $t-s$ (claim 1 of Restricted Lebesgue Measure and Integral Toolkit on a Compact Interval). In particular, for $\omega\in\Omega_0$, where (D4) expresses $\mathcal{T}^a_t(\omega)$ as the integral of the $[0,\Lambda]$-valued map $u\mapsto g^a(u,X_u(\omega))$, the path $t\mapsto\mathcal{T}^a_t(\omega)$ is nondecreasing and continuous with $\mathcal{T}^a_t-\mathcal{T}^a_s\le\Lambda(t-s)$. (P3) Counting paths. For a counting path $c$ with first jump time $\tau_1(c)$ and $s\ge0$: $c(s)\ge1$ if and only if $\tau_1(c)\le s$, by monotonicity and right-continuity of $c$ (Counting Path and Its Jump Times; the set $\{t:c(t)\ge1\}$ is an interval unbounded above whose infimum it contains by right-continuity, or is empty).

Step 1: the one-step estimate. Fix $0\le r<r'\le T$, put $\delta=r'-r$ and $\mathfrak{m}=\Lambda\delta$, fix $D\in\mathfrak{F}_r$ and $F:E\to\mathbb{R}$, and write $\hat{Y}^a=\hat{Y}^{a,r}$ for the residual clocks of (H2); note $\mathfrak{m}=\Lambda(r'-r)\le\Lambda(T-r)=\varrho_r$, so all residual-clock values used below are at times in $[0,\varrho_r]$, where (H2) applies. Every residual path $u\mapsto\hat{Y}^a_u(\omega)$ is itself a counting path (it starts at $0$ and inherits monotonicity, integer values, right-continuity and unit jumps from the counting path of $Y^a$), so (P3) applies to it. Define the frozen consumptions and frozen increments

$$c^a=\int_{[r,r']}g^a(u,X_r)\,du=\sum_{x\in E}\mathbf{1}\{X_r=x\}\int_{[r,r']}g^a(u,x)\,du\in[0,\mathfrak{m}],\qquad G^a=F(\phi^a(X_r))-F(X_r),$$

both $\mathfrak{F}_r$-measurable by (D3), with $|G^a|\le2\|F\|$. Let $B_2=\{\sum_a\hat{Y}^a_\mathfrak{m}\ge2\}$ and $B_1=\{\sum_a\hat{Y}^a_\mathfrak{m}\ge1\}$. For each $a$ let $\xi^a=\tau_1(\hat{Y}^a)\in[0,\infty]$ be the first jump time of the residual clock ($+\infty$ if it never jumps) and let $\zeta^a=\xi^a$ on $\{\xi^a\le\mathfrak{m}\}$ and $\zeta^a=\mathfrak{m}+1$ elsewhere. By (P3), $\{\zeta^a\le s\}=\{\hat{Y}^a_s\ge1\}$ for $0\le s\le\mathfrak{m}$, $\{\zeta^a\le s\}=\{\hat{Y}^a_{\mathfrak{m}}\ge1\}$ for $\mathfrak{m}<s<\mathfrak{m}+1$, $\{\zeta^a\le s\}=\Omega$ for $s\ge\mathfrak{m}+1$, and $\{\zeta^a\le s\}=\emptyset$ for $s<0$; so $\zeta^a$ is a real random variable with $\sigma(\zeta^a)\subseteq\sigma(\hat{Y}^a_u:0\le u\le\varrho_r)$, and, since $c^a\le\mathfrak{m}$, the map $\mathbf{1}\{\hat{Y}^a_{c^a}\ge1\}=\mathbf{1}\{\zeta^a\le c^a\}$ is $\mathcal{F}$-measurable (Arithmetic, Absolute Values, and Pointwise Limits of Measurable Real-Valued Functions). Put $\Phi_u=\mathbf{1}_{\Omega_0}\sum_{a}g^a(u,X_u)\bigl(F(\phi^a(X_u))-F(X_u)\bigr)$, the integrand of claim (a), which is of the form (P1) with $|\Phi_u|\le2|\mathsf{A}|\Lambda\|F\|$. We show

$$\Bigl|\mathbb{E}\bigl[(F(X_{r'})-F(X_r))\mathbf{1}_D\bigr]-\mathbb{E}\Bigl[\mathbf{1}_D\int_{[r,r']}\Phi_u\,du\Bigr]\Bigr|\le C\,\|F\|\,\Lambda^2\delta^2,\qquad C=4|\mathsf{A}|^2(1+|\mathsf{A}|)+|\mathsf{A}|+4|\mathsf{A}|^2 .\tag{1.1}$$

(1a) Pathwise identification on $\Omega_0\setminus B_2$. Fix $\omega\in\Omega_0$. For $u\in[r,r']$, (P2) gives $0\le\mathcal{T}^a_u-\mathcal{T}^a_r\le\mathfrak{m}$, so $N^a_u-N^a_r=\hat{Y}^a_{\mathcal{T}^a_u-\mathcal{T}^a_r}\le\hat{Y}^a_\mathfrak{m}$ by monotonicity of the counting path $\hat{Y}^a$. Now suppose also $\omega\notin B_2$, so that the grand total $N=\sum_aN^a$, a nondecreasing path, satisfies $N_{r'}-N_r\le1$. We claim

$$F(X_{r'})-F(X_r)=\sum_{a\in\mathsf{A}}\mathbf{1}\{\hat{Y}^a_{c^a}\ge1\}\,G^a\qquad\text{on }\Omega_0\setminus B_2.\tag{1.2}$$

Case 1: $N_{r'}=N_r$. Then $N$ is constant on $[r,r']$ by monotonicity, so $N_u=N_{u-}$ for every $u\in(r,r']$, and by (H1) $X_u=X_{u-}$ for every $u\in(r,r']$; since the path is constant on the constancy intervals of (H1) and takes the same value on both sides of each constancy-interval endpoint in $(r,r']$, it is constant on $[r,r']$, so $F(X_{r'})-F(X_r)=0$. Moreover the integrand of $\mathcal{T}^a$ equals $g^a(u,X_r)$ on $[r,r']$, so by (P2) $\mathcal{T}^a_{r'}-\mathcal{T}^a_r=c^a$, whence $\hat{Y}^a_{c^a}=N^a_{r'}-N^a_r=0$ for every $a$, and the right side of (1.2) vanishes too. Case 2: $N_{r'}=N_r+1$. Put $\theta=\inf\{u\in(r,r']:N_u=N_r+1\}$; by right-continuity of $N$ (each $N^a_u=Y^a_{\mathcal{T}^a_u}$ is right-continuous in $u$, the counting path $Y^a$ being right-continuous and $\mathcal{T}^a$ continuous and nondecreasing by (P2)) $N_\theta=N_r+1$ and $\theta>r$, and by monotonicity $N_u=N_r$ for $u\in[r,\theta)$, so $N_{\theta-}=N_r$: thus $N_\theta=N_{\theta-}+1$, and $N_u=N_{u-}$ for every $u\in(r,r']\setminus\{\theta\}$. Since the increments $N^a_{r'}-N^a_r$ are nonnegative integers with sum $1$, exactly one label $b$ has $N^b_{r'}-N^b_r=1$, and $b$ is the unique label with $N^b_\theta\neq N^b_{\theta-}$. By (H1), $X_u=X_{u-}$ for $u\in(r,r']\setminus\{\theta\}$ and $X_\theta=\phi^b(X_{\theta-})$; as in Case 1 the path is constant on $[r,\theta)$ with value $X_r$, so $X_{\theta-}=X_r$, and constant on $[\theta,r']$ with value $\phi^b(X_r)$. Hence $F(X_{r'})-F(X_r)=G^b$. For $a\neq b$: $\hat{Y}^b_\mathfrak{m}\ge N^b_{r'}-N^b_r=1$, so $\hat{Y}^a_\mathfrak{m}=0$ (as $\omega\notin B_2$), and $\hat{Y}^a_{c^a}=0$ since $c^a\le\mathfrak{m}$. For $b$: on $[r,\theta)$ the integrand of $\mathcal{T}^b$ equals $g^b(u,X_r)$, so by (P2) and claim 6 of Restricted Lebesgue Measure and Integral Toolkit on a Compact Interval (the two integrands agree on the co-null set $[r,\theta)$ of $[r,\theta]$) $\mathcal{T}^b_\theta-\mathcal{T}^b_r=\int_{[r,\theta]}g^b(u,X_r)\,du\le c^b$, the inequality by monotonicity of the integral and (P2), the integrand being nonnegative; and $N^b_\theta-N^b_r\ge1$ gives $\hat{Y}^b_{\mathcal{T}^b_\theta-\mathcal{T}^b_r}\ge1$, hence $\hat{Y}^b_{c^b}\ge1$ by monotonicity. Thus the right side of (1.2) equals $G^b$, proving (1.2).

(1b) Probability of $B_2$ and $B_1$. By (H2), $\hat{Y}^a_\mathfrak{m}=\hat{Y}^a_\mathfrak{m}-\hat{Y}^a_0$ has the Poisson distribution with parameter $\mathfrak{m}$ (for $\mathfrak{m}>0$ by the increment assertion of (H2) with $p=1$; for $\mathfrak{m}=0$ because $\hat{Y}^a_0=0$, the residual paths being counting paths, and the Poisson distribution with parameter $0$ is the unit mass at $0$), so $\mathbb{E}[\hat{Y}^a_\mathfrak{m}]=\mathfrak{m}$ and $\mathbb{E}[\hat{Y}^a_\mathfrak{m}(\hat{Y}^a_\mathfrak{m}-1)]=\mathfrak{m}^2$ by Factorial Moments and Moments of Every Order of the Poisson Distribution(a) with $p=1$ and $p=2$. Since $\hat{Y}^a_\mathfrak{m}(\hat{Y}^a_\mathfrak{m}-1)\ge0$ is at least $2$ on $\{\hat{Y}^a_\mathfrak{m}\ge2\}$, Markov's inequality gives $P(\hat{Y}^a_\mathfrak{m}\ge2)\le\mathfrak{m}^2/2$ and $P(\hat{Y}^a_\mathfrak{m}\ge1)\le\mathfrak{m}$. For $a\neq b$ the events $\{\hat{Y}^a_\mathfrak{m}\ge1\}$ and $\{\hat{Y}^b_\mathfrak{m}\ge1\}$ are independent by (H2), so their intersection has probability at most $\mathfrak{m}^2$. As $B_2\subseteq\bigcup_a\{\hat{Y}^a_\mathfrak{m}\ge2\}\cup\bigcup_{a\neq b}(\{\hat{Y}^a_\mathfrak{m}\ge1\}\cap\{\hat{Y}^b_\mathfrak{m}\ge1\})$ and $B_1\subseteq\bigcup_a\{\hat{Y}^a_\mathfrak{m}\ge1\}$, subadditivity (claim 4 of Basic Properties of a Measure, applied to the finite unions padded with empty sets) gives

$$P(B_2)\le|\mathsf{A}|\mathfrak{m}^2/2+|\mathsf{A}|^2\mathfrak{m}^2\le2|\mathsf{A}|^2\mathfrak{m}^2,\qquad P(B_1)\le|\mathsf{A}|\mathfrak{m} .$$

Taking expectations in (1.2) (both sides are bounded, by $2\|F\|$ and $2|\mathsf{A}|\|F\|$, and agree off $B_2\cup(\Omega\setminus\Omega_0)$, a set of probability at most $P(B_2)$),

$$\Bigl|\mathbb{E}\bigl[(F(X_{r'})-F(X_r))\mathbf{1}_D\bigr]-\sum_a\mathbb{E}\bigl[\mathbf{1}\{\hat{Y}^a_{c^a}\ge1\}G^a\mathbf{1}_D\bigr]\Bigr|\le2(1+|\mathsf{A}|)\|F\|\cdot2|\mathsf{A}|^2\mathfrak{m}^2 .\tag{1.3}$$

(1c) Freezing the clock reading. Fix $a$. The random variable $\zeta^a$ introduced above satisfies $\sigma(\zeta^a)\subseteq\sigma(\hat{Y}^a_u:0\le u\le\varrho_r)$ (as recorded there), hence $\sigma(\zeta^a)$ is independent of $\mathfrak{F}_r$ by (H2), and $\mathbf{1}\{\hat{Y}^a_{c^a}\ge1\}=\mathbf{1}\{\zeta^a\le c^a\}$. Write $G^a=G^{a,+}-G^{a,-}$ with $G^{a,\pm}\ge0$ bounded and $\mathfrak{F}_r$-measurable, and for $\pm$ fixed put $\Psi(\omega,v)=\mathbf{1}\{v\le c^a(\omega)\}G^{a,\pm}(\omega)\mathbf{1}_D(\omega)$ for $v\in\mathbb{R}$; $\Psi$ is $\mathfrak{F}_r\otimes\mathcal{B}$-measurable ($\mathcal{B}$ the Borel $\sigma$-algebra), since $(\omega,v)\mapsto c^a(\omega)-v$ is product-measurable by Arithmetic, Absolute Values, and Pointwise Limits of Measurable Real-Valued Functions. Independence Fubini (claim 3, with $\mathcal{G}=\mathfrak{F}_r$ and the independent variable $\zeta^a$) gives

$$\mathbb{E}\bigl[\mathbf{1}\{\zeta^a\le c^a\}G^{a,\pm}\mathbf{1}_D\bigr]=\mathbb{E}\bigl[\psi^a(c^a)\,G^{a,\pm}\mathbf{1}_D\bigr],\qquad \psi^a(s)=P(\zeta^a\le s)=P(\hat{Y}^a_s\ge1)\quad(0\le s\le\mathfrak{m}).$$

Subtracting the two signs, $\mathbb{E}[\mathbf{1}\{\hat{Y}^a_{c^a}\ge1\}G^a\mathbf{1}_D]=\mathbb{E}[\psi^a(c^a)G^a\mathbf{1}_D]$. Now for $0\le s\le\mathfrak{m}$, $\hat{Y}^a_s$ is Poisson with parameter $s$ (as above, including $s=0$), and for an integer $y\ge0$ one has $y-\mathbf{1}\{y\ge1\}=\max(y-1,0)\le y(y-1)/2$ (both sides vanish for $y\in\{0,1\}$, and $y-1\le y(y-1)/2$ for $y\ge2$), so

$$0\le s-\psi^a(s)=\mathbb{E}\bigl[\hat{Y}^a_s-\mathbf{1}\{\hat{Y}^a_s\ge1\}\bigr]\le\tfrac12\mathbb{E}\bigl[\hat{Y}^a_s(\hat{Y}^a_s-1)\bigr]=s^2/2 ,$$

again by Factorial Moments and Moments of Every Order of the Poisson Distribution(a). Hence $|\psi^a(c^a)-c^a|\le\mathfrak{m}^2/2$ everywhere, and

$$\Bigl|\mathbb{E}\bigl[\mathbf{1}\{\hat{Y}^a_{c^a}\ge1\}G^a\mathbf{1}_D\bigr]-\mathbb{E}\bigl[c^aG^a\mathbf{1}_D\bigr]\Bigr|\le\|F\|\mathfrak{m}^2 .\tag{1.4}$$

(1d) Unfreezing the state. Put $I^a=\int_{[r,r']}\mathbf{1}_{\Omega_0}g^a(u,X_u)\bigl(F(\phi^a(X_u))-F(X_u)\bigr)\,du$, so that $\sum_aI^a=\int_{[r,r']}\Phi_u\,du$, and note $c^aG^a=\int_{[r,r']}g^a(u,X_r)\bigl(F(\phi^a(X_r))-F(X_r)\bigr)\,du$. On $\Omega_0\setminus B_1$ no counter jumps in $(r,r']$ (by (1a), $N^a_{r'}-N^a_r\le\hat{Y}^a_\mathfrak{m}=0$), so as in Case 1 the path $X$ is constant on $[r,r']$ and $I^a=c^aG^a$. Both $I^a$ and $c^aG^a$ are bounded by $2\|F\|\mathfrak{m}$ (by (P2)), so

$$\Bigl|\mathbb{E}\bigl[c^aG^a\mathbf{1}_D\bigr]-\mathbb{E}\bigl[I^a\mathbf{1}_D\bigr]\Bigr|\le4\|F\|\mathfrak{m}\,P(B_1)\le4|\mathsf{A}|\,\|F\|\mathfrak{m}^2 .\tag{1.5}$$

Combining (1.3), (1.4) summed over $a$, and (1.5) summed over $a$, and recalling $\mathfrak{m}=\Lambda\delta$, gives (1.1) with $C=4|\mathsf{A}|^2(1+|\mathsf{A}|)+|\mathsf{A}|+4|\mathsf{A}|^2$.

Step 2: summation over a partition (claim (a)). Let $0\le r<t\le T$, $D\in\mathfrak{F}_r$, $F:E\to\mathbb{R}$, and let $n\ge1$ be a natural number; put $r_k=r+k(t-r)/n$ for $k\in\{0,\dots,n\}$. Since $(\mathfrak{F}_s)$ is a filtration, $D\in\mathfrak{F}_r\subseteq\mathfrak{F}_{r_k}$ for every $k$, so Step 1 applies on each $[r_k,r_{k+1}]$ with the same $D$ and $\delta=(t-r)/n$. Summing (1.1) over $k$, the left sides telescope, $\sum_k\mathbb{E}[(F(X_{r_{k+1}})-F(X_{r_k}))\mathbf{1}_D]=\mathbb{E}[(F(X_t)-F(X_r))\mathbf{1}_D]$, while by (P2) $\sum_k\int_{[r_k,r_{k+1}]}\Phi_u\,du=\int_{[r,t]}\Phi_u\,du$ pointwise on $\Omega$ (the map $u\mapsto\Phi_u(\omega)$ is bounded and measurable, being a section of a product-measurable map by (P1)). Therefore

$$\Bigl|\mathbb{E}\bigl[(F(X_t)-F(X_r))\mathbf{1}_D\bigr]-\mathbb{E}\Bigl[\mathbf{1}_D\int_{[r,t]}\Phi_u\,du\Bigr]\Bigr|\le n\cdot C\|F\|\Lambda^2\frac{(t-r)^2}{n^2}=\frac{C\|F\|\Lambda^2(t-r)^2}{n}.$$

The left side does not depend on $n$; letting $n\to\infty$ shows it is $0$, which is claim (a) for $r<t$. For $r=t$ both sides of (a) are $0$. The measurability assertion in (a) is (P1) with $H(u,x)=\sum_ag^a(u,x)(F(\phi^a(x))-F(x))$.

Step 3: claim (b). Fix $r\in[0,T)$ and $D\in\mathfrak{F}_r$. For $x\neq y$ the map $u\mapsto q_u(x,y)$ is a finite sum of the measurable maps $u\mapsto g^a(u,x)$, hence measurable (Arithmetic, Absolute Values, and Pointwise Limits of Measurable Real-Valued Functions), and $\sum_{y\neq x}q_u(x,y)\le\sum_ag^a(u,x)\le|\mathsf{A}|\Lambda$, each $a$ contributing to at most one $y$. Since $P(\Omega_0)=1$, $\mu^D_u(x)=\mathbb{E}[\mathbf{1}_D\mathbf{1}_{\Omega_0}\mathbf{1}\{X_u=x\}]$, which is a measurable function of $u\in[0,T]$ with values in $[0,1]$ by (P1) (with $H(u,x')=\mathbf{1}\{x'=x\}$); restricted to $[r,T]$ it is measurable for $\mathcal{B}_{[r,T]}$, the preimages being intersections of members of $\mathcal{B}_{[0,T]}$ with $[r,T]$. This is condition (i) of Uniqueness for the Forward Equation of a Bounded Jump-Rate Family on a Finite Set. For condition (ii), let $F:E\to\mathbb{R}$ and $t\in[r,T]$. Then $\mathbb{E}[F(X_t)\mathbf{1}_D]=\sum_xF(x)P(D\cap\{X_t=x\})=\mu^D_t(F)$ with the pairing $\mu(F)=\sum_x\mu(x)F(x)$ of Uniqueness for the Forward Equation of a Bounded Jump-Rate Family on a Finite Set, and likewise at time $r$; and by (P1),

$$\mathbb{E}\Bigl[\mathbf{1}_D\int_{[r,t]}\Phi_u\,du\Bigr]=\int_{[r,t]}\mathbb{E}\bigl[\mathbf{1}_D\Phi_u\bigr]\,du=\int_{[r,t]}\sum_{x\in E}\mu^D_u(x)\sum_{a\in\mathsf{A}}g^a(u,x)\bigl(F(\phi^a(x))-F(x)\bigr)\,du=\int_{[r,t]}\mu^D_u(\mathcal{L}_uF)\,du,$$

where the last equality regroups the labels $a$ according to $y=\phi^a(x)$: the labels with $\phi^a(x)=x$ contribute $0$, and for $y\neq x$ the labels with $\phi^a(x)=y$ contribute $q_u(x,y)(F(y)-F(x))$, so the inner sum is $\sum_{y\neq x}q_u(x,y)(F(y)-F(x))=\mathcal{L}_uF(x)$ as defined in Uniqueness for the Forward Equation of a Bounded Jump-Rate Family on a Finite Set. Claim (a) now reads $\mu^D_t(F)=\mu^D_r(F)+\int_{[r,t]}\mu^D_u(\mathcal{L}_uF)\,du$, which is condition (ii).