Post-Exit Comparison at a Stopping Time for the Recentred N-Agent Cost under To-Go Value Regularity

Adopt the setting, hypotheses and notation of the pathwise tracking lemma: the affine-controlled transition-rate family $(\beta_{0},\beta_{1})$ on $l$ states with control set $\mathcal{A}\subseteq\mathbb{R}^{m}$, its transition-rate family $\beta$ with rate bound $B$, aggregate state drift $b$, projected drift $\hat{b}$ and state-Lipschitz constant $\Lambda_{b}$; the observation-rate family $\tilde{\beta}$; the horizon $T>0$; the population cost data $(L,G)$, convex in the control on $\mathcal{A}$; the probability simplex $\Delta^{l}$; the set $\mathcal{U}_{\mathcal{A}}$ of $\mathcal{A}$-valued controls; the mean-field flow $S(z_{0},\xi)$ of claim 2 of the flow stability lemma (the two-argument flow map, distinguished throughout from the trajectory $S$ of the mean-field trajectory pair below by the presence of its arguments); the mean-field cost $F$; the $N$-agent driving system $(\Omega,\mathcal{F},P)$; the $\mathcal{A}$-valued policy $h$; the solution with regular event $\Omega_{0}$, empirical state measure $\Sigma_{t}$, control $\alpha_{t}$ and system filtration $(\mathcal{F}^{\mathrm{sys}}_{t})_{t\in[0,T]}$; the realized control $\hat{\alpha}$ of the realized-control lemma; and the martingale part $M=(M^{1},\dots,M^{l})$ of the martingale decomposition theorem. The hypothesis (LipC) of the tracking lemma, with constants $K_{L}$ and $K_{G}$, is assumed only in claim 3.

Let further $(S,A)$ be a mean-field trajectory pair for $\beta$ with horizon $T$, let $(U,V,\bar{\beta})$ be a twice continuously differentiable extension of $\beta$ with derivative bound $K$ and extended aggregate state drift $\bar{b}$, let $(U_{c},\bar{L},\bar{G})$ be a twice continuously differentiable extension of $(L,G)$ with second-derivative bound $K_{c}$, and let $P$ be a stationary co-state for these data, so that the setting of the first-order expansion lemma for the recentred $N$-agent cost is in force; adopt from it the fixed constant $C_{P}$, the mean-field Hamiltonian $\mathcal{H}_{t}(\Sigma,\alpha)$ and its state derivative coefficients $\partial_{\gamma}\mathcal{H}_{t}(\Sigma,\alpha)$, the quantities $\mathcal{D}_{t}$ and $\mathcal{D}_{G}$ with their bounds $C_{\mathcal{D}}$ and $C_{\mathcal{D}G}$, the fluctuation process $\mathfrak{s}_{t}=\sqrt{N}\,(\Sigma_{t}-S_{t})$, and the abbreviation $y_{t}=\Sigma_{t}-S_{t}$. Adopt also the setting and notation of the time-shift lemma and of the to-go comparison lemma for these data (the affine family, cost data and extensions being the same, and the triple $(S,A,P)$ that of this statement), including the horizon-$\theta$ instance notation superscripted by $[\theta]$ and, in claim 3, the hypothesis (TG) with constants $\varepsilon_{tg}$ and $C_{tg}$.

Stopping times are those of $(\mathcal{F}^{\mathrm{sys}}_{t})_{t\in[0,T]}$; for a stopping time $\varsigma$, $\mathcal{F}_{\varsigma}$ denotes the $\sigma$-algebra of events prior to $\varsigma$, sampled functions such as $M^{\gamma}_{\varsigma}$ and $\Sigma_{\varsigma}$ are the sampled functions, and $\mathbf{1}_{\{t<\varsigma\}}$ is the pre-stopping-time indicator. For a family $X=(X_{t})_{t\in[0,T]}$, a point $\omega\in\Omega$ at which the path $t\mapsto X_{t}(\omega)$ is measurable and bounded, and a stopping time $\varsigma$, write $\int_{[\varsigma,T]}X_{t}\,dt$ for the Lebesgue integral of the path over $[\varsigma(\omega),T]$ when $\varsigma(\omega)<T$, and $0$ when $\varsigma(\omega)=T$. Write $\mathbf{1}_{D}$ for the indicator of a set $D$, $\mathbb{E}$ for the expectation, $|\cdot|$ for the Euclidean norm, and $e^{x}$ for the real exponential function. Adopt the constants $c_{M}=6\,l^{2}(2(l-1))^{4}$ and $\kappa_{T}=BT+(BT)^{2}$ of the restricted moments lemma, and for a nonnegative real $x$ let $x^{1/4}$ and $x^{3/4}$ be as defined there. Define

$$C^{\vee}_{tg}=\max\Big(C_{tg},\ \tfrac{l\,K_{c}}{2}\Big),\qquad C_{ns}=2\,l\,(K_{L}T+K_{G})\,e^{l\,\Lambda_{b}T}\,\big(4\,c_{M}\,\kappa_{T}\big)^{1/4}.$$

Let $\varsigma$ be a stopping time. Define, at every $\omega\in\Omega_{0}$,

$$R^{M,\varsigma}=-\sum_{\gamma=1}^{l}P^{\gamma}_{T}\,\big(M^{\gamma}_{T}-M^{\gamma}_{\varsigma}\big)+\int_{[\varsigma,T]}\sum_{\gamma=1}^{l}\partial_{\gamma}\mathcal{H}_{t}(S_{t},A_{t})\,\big(M^{\gamma}_{t}-M^{\gamma}_{\varsigma}\big)\,dt ,$$

and $R^{M,\varsigma}=0$ at every $\omega\notin\Omega_{0}$.

Then the following hold.

1. (Anchored pathwise identity.) For every $\omega\in\Omega_{0}$, the paths on $[0,T]$ of $t\mapsto L(\Sigma_{t},\alpha_{t})$, $t\mapsto\mathcal{D}_{t}$, $t\mapsto y^{\gamma}_{t}$ and $t\mapsto M^{\gamma}_{t}$ are measurable and bounded, all integrals below exist, and, writing $r=\varsigma(\omega)$,

$$\int_{[\varsigma,T]}L(\Sigma_{t},\alpha_{t})\,dt+G(\Sigma_{T})-\int_{[\varsigma,T]}L(S_{t},A_{t})\,dt-G(S_{T})+\sum_{\gamma=1}^{l}P^{\gamma}_{r}\,y^{\gamma}_{r}=\int_{[\varsigma,T]}\mathcal{D}_{t}\,dt+\mathcal{D}_{G}+R^{M,\varsigma}.$$

Moreover the functions $\mathbf{1}_{\Omega_{0}}\int_{[\varsigma,T]}\mathcal{D}_{t}\,dt$ (set to $0$ off $\Omega_{0}$), $R^{M,\varsigma}$, $\mathbf{1}_{\Omega_{0}}M^{\gamma}_{\varsigma}$, $\mathbf{1}_{\Omega_{0}}y^{\gamma}_{\varsigma}$, $\mathbf{1}_{\Omega_{0}}|\mathfrak{s}_{\varsigma}|^{2}$ and $\omega\mapsto P^{\gamma}_{\varsigma(\omega)}$ are bounded random variables.

2. (Vanishing of the anchored martingale remainder.) For every event $D\in\mathcal{F}_{\varsigma}$,

$$\mathbb{E}\big[\mathbf{1}_{D}\,R^{M,\varsigma}\big]=0 .$$

3. (Post-exit comparison.) Assume (LipC), assume $[A]\in\mathcal{M}^{*}_{S_{0}}$ (the class of $A$ is an optimal mean-field control from $S_{0}$, as in claim 5 of the time-shift lemma), and assume hypothesis (TG) of the to-go comparison lemma with constants $\varepsilon_{tg}>0$ and $C_{tg}\ge0$. Let $D\in\mathcal{F}_{\varsigma}$ be an event with $D\subseteq\Omega_{0}$ such that $|\Sigma_{\varsigma(\omega)}(\omega)-S_{\varsigma(\omega)}|\le\varepsilon_{tg}$ for every $\omega\in D$. Then

$$\mathbb{E}\Big[\mathbf{1}_{D}\Big(N\int_{[\varsigma,T]}\mathcal{D}_{t}\,dt+N\,\mathcal{D}_{G}\Big)\Big]\ \ge\ -\,C^{\vee}_{tg}\,\mathbb{E}\big[\mathbf{1}_{D}\,|\mathfrak{s}_{\varsigma}|^{2}\big]-C_{ns}\,\sqrt{N}\,P(D)^{3/4},$$

all expectations being defined, the integrands being bounded random variables by claim 1 and part (a) of the first-order expansion lemma.