First-Order Expansion of the Mean-Field Cost about a Stationary Mean-Field Triple and Its Quadratic Lower Bound

Let $l$, $m$, $\mathcal{A}$, $\beta$ with rate bound $B$, $(U,V,\bar{\beta})$ with derivative bound $K$, $\bar{b}$, $(L,G)$, $(U_c,\bar{L},\bar{G})$ with second-derivative bound $K_c$ (the open set of the cost extension, written $W$ in that definition, is written $U_c$ here), $T>0$, and $(S,A)$ be as in the definition of a stationary mean-field triple, and let $P$ be a stationary co-state for these data. Adopt the coordinate and partial-derivative notation $\partial_i$, $\partial_j\partial_i$ of the extension definitions; the partial derivatives of orders one and two of each $\bar{b}^\delta$ exist and are continuous on $U\times V$ by part (i) of the regularity of the extended aggregate state drift, and those of $\bar{L}$ and $\bar{G}$ by clause 2 of the cost extension definition together with clauses 1 and 2 of the $C^k$ definition. Let $b$ be the aggregate state drift of $\beta$, which agrees with $\bar{b}$ on $\Delta^l\times\mathcal{A}$ by part (i) of the regularity lemma, where $\Delta^l$ is the probability simplex, and let $J^{MF}[(S),(A)]$ be the mean-field cost of $(S,A)$ under $(L,G)$. Write $\mathbb{R}$ for the real numbers, $|\cdot|$ for the Euclidean norm (Euclidean distance to the origin), and $\int_{[0,t]}\cdot\,ds$ for the Lebesgue integral over the compact interval $[0,t]$ (taken to be $0$ for $t=0$), the Riemann integrals of continuous functions appearing in the cited definitions agreeing with their Lebesgue integrals by claim 3 of the same toolkit. Throughout, a real-valued function on a subinterval $I$ of the real numbers $\mathbb{R}$ is called continuous on $I$ when it is continuous relative to $I$, both $I$ and the codomain $\mathbb{R}$ carrying the metric of the real line. Fix a real $C_P$ with $\sum_{\delta=1}^{l}|P^\delta_t|\le C_P$ for all $t\in[0,T]$, which exists by clause 1 of the co-state definition and the extreme value theorem. Assume that $\mathcal{A}$ is compact for the topology determined by the Euclidean distance; in (b) and (c) assume moreover that $\mathcal{A}$ is convex (conclusion (a) does not use convexity).

Define the mean-field Hamiltonian by

$$\mathcal{H}_t(\Sigma,\alpha)=\bar{L}(\Sigma,\alpha)-\sum_{\delta=1}^{l}P^\delta_t\,\bar{b}^\delta(\Sigma,\alpha)\qquad(t\in[0,T],\ \Sigma\in U\cap U_c,\ \alpha\in V),$$

and, for $\gamma\in\{1,\dots,l\}$, its state derivative coefficients

$$\partial_\gamma\mathcal{H}_t(\Sigma,\alpha)=\partial_\gamma\bar{L}(\Sigma,\alpha)-\sum_{\delta=1}^{l}P^\delta_t\,\partial_\gamma\bar{b}^\delta(\Sigma,\alpha),$$

the left-hand side being notation for the right-hand side. The function $a\mapsto\mathcal{H}_t(S_t,a)$ is the Hamiltonian $\mathcal{H}_t(a)$ of Quadratic Growth of the Mean-Field Hamiltonian in the Control along a Stationary Mean-Field Triple.

An admissible pair is a pair of maps $x:[0,T]\to\Delta^l$ and $a:[0,T]\to\mathcal{A}$ such that every component $t\mapsto x^\gamma_t$ is continuous on $[0,T]$, every component $t\mapsto a^j_t$ is measurable with respect to the trace Borel $\sigma$-algebra on $[0,T]$ and the Borel $\sigma$-algebra on the real line, and

$$x^\gamma_t=x^\gamma_0+\int_{[0,t]}b^\gamma(x_s,a_s)\,ds\qquad\text{for all }t\in[0,T]\text{ and }\gamma\in\{1,\dots,l\};$$

here the integrand is measurable, as a sequentially continuous function of measurable maps by measurability of continuous functions of measurable maps ($\bar{b}^\gamma$ being continuous on the open set $U\times V$), and bounded in absolute value by $2(l-1)B$: by the definition of the aggregate state drift, $|b^\gamma(\Sigma,\alpha)|\le\sum_{\sigma\neq\gamma}\big(\Sigma^\sigma\beta(\sigma,\gamma,\Sigma,\alpha)+\Sigma^\gamma\beta(\gamma,\sigma,\Sigma,\alpha)\big)\le B\sum_{\sigma\neq\gamma}\Sigma^\sigma+(l-1)B\,\Sigma^\gamma\le lB\le2(l-1)B$ for $(\Sigma,\alpha)\in\Delta^l\times\mathcal{A}$, since $0\le\beta\le B$ by the definition of a transition-rate family, the coordinates of $\Sigma$ are nonnegative with sum $1$, and $l\ge2$; so the integral exists. The pair $(S,A)$ itself is admissible by the trajectory-pair definition. For an admissible pair put $y_t=x_t-S_t\in\mathbb{R}^l$ and define its cost

$$J[(x),(a)]=\int_{[0,T]}L(x_t,a_t)\,dt+G(x_T),$$

whose integrand is measurable by the same composition lemma ($L$ being sequentially continuous on $\Delta^l\times\mathbb{R}^m$ by clause 1 of the definition of population cost data) and bounded by claim 1 of the boundedness of cost data over a compact control set; when $(x,a)=(S,A)$ this is $J^{MF}[(S),(A)]$.

Let $(x,a)$ be an admissible pair. Then:

(a) (Exact first-order identity.) All integrands below are measurable and bounded on $[0,T]$, and

$$J[(x),(a)]-J^{MF}[(S),(A)]=\int_{[0,T]}\Big(\mathcal{H}_t(x_t,a_t)-\mathcal{H}_t(S_t,A_t)-\sum_{\gamma=1}^{l}\partial_\gamma\mathcal{H}_t(S_t,A_t)\,y^\gamma_t\Big)dt+\Big(\bar{G}(x_T)-\bar{G}(S_T)-\sum_{\gamma=1}^{l}\partial_\gamma\bar{G}(S_T)\,y^\gamma_T\Big)-\sum_{\gamma=1}^{l}P^\gamma_0\,y^\gamma_0 .$$

(b) (Pointwise bounds.) Put $M_2=K_c+3\,l\,K\,C_P$, $C_1=l\sqrt{l+m}\,M_2$, and $C_2=\tfrac{1}{2}(l+m)M_2$, with $\sqrt{\cdot}$ the nonnegative square root; $M_2$ and $C_2$ are the constants $M_2$ and $C_{\mathcal{H}}$ of Quadratic Growth of the Mean-Field Hamiltonian in the Control along a Stationary Mean-Field Triple for the same choice of $C_P$. For every $t\in[0,T]$, every $\Sigma\in\Delta^l$, and every $\alpha\in\mathcal{A}$,

$$\mathcal{H}_t(\Sigma,\alpha)-\mathcal{H}_t(S_t,A_t)-\sum_{\gamma=1}^{l}\partial_\gamma\mathcal{H}_t(S_t,A_t)(\Sigma^\gamma-S^\gamma_t)\ \ge\ \big(\mathcal{H}_t(S_t,\alpha)-\mathcal{H}_t(S_t,A_t)\big)-C_1\,|\Sigma-S_t|\,|\alpha-A_t|-C_2\,|\Sigma-S_t|^2 ,$$

and for every $\Sigma\in\Delta^l$,

$$\Big|\bar{G}(\Sigma)-\bar{G}(S_T)-\sum_{\gamma=1}^{l}\partial_\gamma\bar{G}(S_T)(\Sigma^\gamma-S^\gamma_T)\Big|\le\tfrac{1}{2}\,l\,K_c\,|\Sigma-S_T|^2 .$$

(c) (Quadratic lower bound.) Assume in addition that there is a real $r_0>0$ with $\mathcal{H}_t(S_t,\alpha)-\mathcal{H}_t(S_t,A_t)\ge r_0|\alpha-A_t|^2$ for every $t\in[0,T]$ and $\alpha\in\mathcal{A}$ (as furnished by conclusion (d) of Quadratic Growth of the Mean-Field Hamiltonian in the Control along a Stationary Mean-Field Triple under its hypotheses). Then

$$J[(x),(a)]-J^{MF}[(S),(A)]+\sum_{\gamma=1}^{l}P^\gamma_0\,y^\gamma_0\ \ge\ \frac{r_0}{2}\int_{[0,T]}|a_t-A_t|^2\,dt-\Big(C_2+\frac{C_1^2}{2r_0}\Big)\int_{[0,T]}|y_t|^2\,dt-\frac{l\,K_c}{2}\,|y_T|^2 ,$$

all integrals being Lebesgue integrals of bounded measurable functions.