Proof of First-Order Expansion of the Mean-Field Cost about a Stationary Mean-Field Triple and Its Quadratic Lower Bound

All integrals over subintervals of $[0,T]$ are Lebesgue integrals as in the statement, and we use linearity and monotonicity of the integral freely. We first record the standing facts. (F1) $|\partial_j\partial_i\bar{L}(z)|\le K_c$ for all $i,j\in\{1,\dots,l+m\}$ and $z\in U_c\times\mathbb{R}^m$, and $|\partial_{\gamma'}\partial_\gamma\bar{G}(\Sigma)|\le K_c$ for all $\gamma,\gamma'$ and $\Sigma\in U_c$ (clause 3 of the cost extension definition); $|\partial_j\partial_i\bar{b}^\delta(z)|\le3lK$ for all $i,j,\delta$ and $z\in\Delta^l\times V$ (part (iii) of the regularity lemma). (F2) For $\Sigma\in\Delta^l$ and $\alpha\in\mathcal{A}$, the segment $\{(S_t+\tau(\Sigma-S_t),\alpha):\tau\in[0,1]\}$ lies in $\Delta^l\times\mathcal{A}$, since $\tau\Sigma+(1-\tau)S_t$ has nonnegative coordinates summing to $1$; the segment $\{(S_t,A_t+\tau(\alpha-A_t)):\tau\in[0,1]\}$ lies in $\{S_t\}\times\mathcal{A}$ when $\mathcal{A}$ is convex, as is assumed in (b) and (c) — this clause of (F2) is used only in Step 4; and the segment $\{S_T+\tau(\Sigma-S_T):\tau\in[0,1]\}$ lies in $\Delta^l$. All of these lie in $U_c\times\mathbb{R}^m$ (respectively $U_c$) and in $\Delta^l\times V\subseteq U\times V$, because $\Delta^l\subset U_c$, $\Delta^l\subset U$ and $\mathcal{A}\subseteq V$ by the extension definitions. (F3) By clause 1 of the cost extension definition and part (i) of the regularity lemma, $L=\bar{L}$ on $\Delta^l\times\mathbb{R}^m$, $G=\bar{G}$ on $\Delta^l$, and $b^\delta=\bar{b}^\delta$ on $\Delta^l\times\mathcal{A}$.

Step 1: measurability and boundedness. Let $(x,a)$ be admissible. The map $t\mapsto(x_t,a_t)\in\mathbb{R}^{l+m}$ has measurable components (continuous components being measurable by claim 3 of the integral toolkit), so by measurability of sequentially continuous functions of measurable maps the functions $t\mapsto\bar{L}(x_t,a_t)$, $t\mapsto\bar{b}^\delta(x_t,a_t)$, and products of these with the continuous functions $t\mapsto P^\delta_t$ are measurable ($\bar{L}$ and $\bar{b}^\delta$ being continuous at every point of their open domains by clause 1 of the $C^k$ definition, and products of continuous functions of measurable maps being again such functions). They are bounded: $|\bar{L}(x_t,a_t)|=|L(x_t,a_t)|$ is bounded, by claim 1 of the boundedness of cost data, $|\bar{b}^\delta(x_t,a_t)|=|b^\delta(x_t,a_t)|\le2(l-1)B$ by the bound on $b$ over $\Delta^l\times\mathcal{A}$ derived in the statement, and $|P^\delta_t|\le C_P$. The functions $t\mapsto\partial_\gamma\mathcal{H}_t(S_t,A_t)$, $t\mapsto\mathcal{H}_t(S_t,A_t)$, and $t\mapsto y^\gamma_t$ are continuous on $[0,T]$ — compositions of the continuous maps $t\mapsto(S_t,A_t)$ and $t\mapsto P^\delta_t$ with continuous functions, by continuity of compositions and continuity of sums and products, the metric and Euclidean notions agreeing by claim 1 of the agreement lemma — hence measurable and bounded by the extreme value theorem. Consequently every integrand appearing in (a), (c), and below is measurable and bounded, hence integrable over $[0,T]$ and over each $[0,t]$.

Step 2: forward integral forms of the co-state and of $y$. For $\gamma\in\{1,\dots,l\}$ put $f^\gamma(s)=\partial_\gamma\mathcal{H}_s(S_s,A_s)$ and $g^\gamma(s)=b^\gamma(x_s,a_s)-b^\gamma(S_s,A_s)$ for $s\in[0,T]$. The function $-f^\gamma$ is exactly the integrand of clause 2 of the co-state definition, which is continuous on $[0,T]$ by that clause; by claim 3 of the integral toolkit (applied to the restriction of $-f^\gamma$ to $[t,T]$, continuous by claim 1 of restriction stability) the Riemann integral in clause 2 equals the Lebesgue integral over $[t,T]$ for $0\le t<T$. Fix $t\in(0,T)$ and let $\phi_1,\phi_2,\phi_3:[0,T]\to\mathbb{R}$ be the functions equal to $-f^\gamma$ on $[0,t]$, on $(t,T]$, and on $[t,T]$ respectively, and to $0$ elsewhere on $[0,T]$; they are measurable and bounded, $-f^\gamma=\phi_1+\phi_2$ on $[0,T]$, and $\phi_2$ and $\phi_3$ agree outside the single point $t$, a set of Lebesgue measure zero, so $\int_{[0,T]}\phi_2=\int_{[0,T]}\phi_3$ by claim 2 of the null-set lemma. By claim 2 of the toolkit, the Lebesgue integral of a function over $[0,t]$, over $[t,T]$, or over $[0,T]$ equals the integral over $\mathbb{R}$ of its zero extension, and the zero extensions of $\phi_1$ and of the restriction of $-f^\gamma$ to $[0,t]$ coincide, as do those of $\phi_3$ and of the restriction of $-f^\gamma$ to $[t,T]$. Hence, by linearity, for every $t\in[0,T]$,

$$\int_{[0,T]}(-f^\gamma)(s)\,ds=\int_{[0,t]}(-f^\gamma)(s)\,ds+\int_{[t,T]}(-f^\gamma)(s)\,ds ,$$

with the convention that the middle term is $0$ for $t=0$ and the last is $0$ for $t=T$ (the latter being the convention of clause 2 of the co-state definition); for $t=0$ and for $t=T$ the identity is immediate from these conventions, both sides reducing to $\int_{[0,T]}(-f^\gamma)$. Clause 2 of the co-state definition at $t$ and at $0$ therefore gives

$$P^\gamma_t-P^\gamma_0=\int_{[t,T]}(-f^\gamma)(s)\,ds-\int_{[0,T]}(-f^\gamma)(s)\,ds=\int_{[0,t]}f^\gamma(s)\,ds ,\qquad\text{and}\qquad P^\gamma_T=-\partial_\gamma\bar{G}(S_T).$$

Likewise, by clause 2 of the trajectory-pair definition and claim 3 of the toolkit, $S^\gamma_t=S^\gamma_0+\int_{[0,t]}b^\gamma(S_s,A_s)\,ds$, so by the admissibility equation and linearity

$$y^\gamma_t=y^\gamma_0+\int_{[0,t]}g^\gamma(s)\,ds\qquad(t\in[0,T]).$$

Step 3: proof of (a). By linearity, and since the Riemann integral defining the mean-field cost equals the Lebesgue integral (claim 3 of the toolkit),

$$J[(x),(a)]-J^{MF}[(S),(A)]=\int_{[0,T]}\big(L(x_t,a_t)-L(S_t,A_t)\big)dt+G(x_T)-G(S_T).$$

By (F3) and the definition of $\mathcal{H}_t$, at every $t$ we have $L(x_t,a_t)=\mathcal{H}_t(x_t,a_t)+\sum_\delta P^\delta_t\,b^\delta(x_t,a_t)$ and $L(S_t,A_t)=\mathcal{H}_t(S_t,A_t)+\sum_\delta P^\delta_t\,b^\delta(S_t,A_t)$, so the integrand equals $\mathcal{H}_t(x_t,a_t)-\mathcal{H}_t(S_t,A_t)+\sum_\delta P^\delta_t\,g^\delta(t)$. For each $\delta$, integration by parts for indefinite Lebesgue integrals applies with $u_t=P^\delta_t$, $f=f^\delta$, $v_t=y^\delta_t$, $g=g^\delta$ (both $f^\delta$ and $g^\delta$ are integrable by Step 1, and the integral forms are those of Step 2); part (ii) of that lemma, which also records the integrability of $f^\delta y^\delta$ and of $P^\delta g^\delta$, together with linearity, gives

$$\int_{[0,T]}P^\delta_t\,g^\delta(t)\,dt=P^\delta_T\,y^\delta_T-P^\delta_0\,y^\delta_0-\int_{[0,T]}f^\delta(t)\,y^\delta_t\,dt=-\partial_\delta\bar{G}(S_T)\,y^\delta_T-P^\delta_0\,y^\delta_0-\int_{[0,T]}\partial_\delta\mathcal{H}_t(S_t,A_t)\,y^\delta_t\,dt .$$

Summing over $\delta$, substituting, and using $G=\bar{G}$ on $\Delta^l$ from (F3) yields the identity of (a).

Step 4: proof of (b). Fix $t$, $\Sigma\in\Delta^l$, $\alpha\in\mathcal{A}$, and abbreviate $w=\Sigma-S_t$. Algebraically,

$$\mathcal{H}_t(\Sigma,\alpha)-\mathcal{H}_t(S_t,A_t)-\sum_\gamma\partial_\gamma\mathcal{H}_t(S_t,A_t)w^\gamma=\mathrm{I}+\mathrm{II}+\mathrm{III},$$

where $\mathrm{I}=\mathcal{H}_t(S_t,\alpha)-\mathcal{H}_t(S_t,A_t)$, $\mathrm{II}=\mathcal{H}_t(\Sigma,\alpha)-\mathcal{H}_t(S_t,\alpha)-\sum_\gamma\partial_\gamma\mathcal{H}_t(S_t,\alpha)w^\gamma$, and $\mathrm{III}=\sum_\gamma\big(\partial_\gamma\mathcal{H}_t(S_t,\alpha)-\partial_\gamma\mathcal{H}_t(S_t,A_t)\big)w^\gamma$.

Term II. Apply part (ii) of the multivariate Taylor expansion with $n=l+m$, $x=(S_t,\alpha)$, $y=(\Sigma,\alpha)$ — the segment lying in $\Delta^l\times\mathcal{A}$ by (F2), and the difference vector having components $w^\gamma$ in the first $l$ coordinates and $0$ in the last $m$, with Euclidean length $|w|$ — to $f=\bar{L}$ on the open set $U_c\times\mathbb{R}^m$ with the second-derivative bound of that lemma taken to be $K_c$, and to each $f=\bar{b}^\delta$ on the open set $U\times V$ with that bound taken to be $3lK$, the bounds (F1) holding on the segment (the symbol $M_2$ is reserved for the constant $K_c+3lKC_P$ of the statement). Multiplying the $\bar{b}^\delta$ estimates by $-P^\delta_t$, adding, and using $\sum_\delta|P^\delta_t|\le C_P$ gives $|\mathrm{II}|\le\tfrac{1}{2}(l+m)(K_c+3lKC_P)|w|^2=C_2|w|^2$.

Term III. For each $\gamma$, the function $\partial_\gamma\bar{L}$ is of class $C^1$ on $U_c\times\mathbb{R}^m$ by clause 2 of the $C^k$ definition (as $\bar{L}$ is of class $C^2$), with partial derivatives $\partial_i\partial_\gamma\bar{L}$ bounded by $K_c$ there; and $\partial_\gamma\bar{b}^\delta$ is of class $C^1$ on $U\times V$ by part (i) of the regularity lemma, with partial derivatives bounded by $3lK$ on $\Delta^l\times V$. Part (i) of the Taylor lemma along the segment from $(S_t,A_t)$ to $(S_t,\alpha)$, which lies in $\{S_t\}\times\mathcal{A}$ by (F2) and has length $|\alpha-A_t|$, gives $|\partial_\gamma\bar{L}(S_t,\alpha)-\partial_\gamma\bar{L}(S_t,A_t)|\le\sqrt{l+m}\,K_c\,|\alpha-A_t|$ and $|\partial_\gamma\bar{b}^\delta(S_t,\alpha)-\partial_\gamma\bar{b}^\delta(S_t,A_t)|\le\sqrt{l+m}\,3lK\,|\alpha-A_t|$, whence $|\partial_\gamma\mathcal{H}_t(S_t,\alpha)-\partial_\gamma\mathcal{H}_t(S_t,A_t)|\le\sqrt{l+m}\,M_2\,|\alpha-A_t|$ for every $\gamma$. Since $|w^\gamma|\le|w|$ for each of the $l$ coordinates, $|\mathrm{III}|\le l\sqrt{l+m}\,M_2\,|\alpha-A_t|\,|w|=C_1|\alpha-A_t||w|$.

Combining, the left-hand side is at least $\mathrm{I}-C_1|w||\alpha-A_t|-C_2|w|^2$, which is the first inequality of (b). For the second, apply part (ii) of the Taylor lemma to $f=\bar{G}$ on the open set $U_c$, of class $C^2$ by clause 2 of the cost extension definition, with $n=l$, $x=S_T$, $y=\Sigma$, the segment lying in $\Delta^l$ by (F2), and with the second-derivative bound of that lemma taken to be $K_c$ by (F1).

Step 5: proof of (c). By (b) applied at each $t$ with $\Sigma=x_t$ and $\alpha=a_t$, and by the hypothesis on $r_0$, the integrand of (a) satisfies, at every $t\in[0,T]$,

$$\mathcal{H}_t(x_t,a_t)-\mathcal{H}_t(S_t,A_t)-\sum_\gamma\partial_\gamma\mathcal{H}_t(S_t,A_t)y^\gamma_t\ \ge\ r_0|a_t-A_t|^2-C_1|y_t||a_t-A_t|-C_2|y_t|^2\ \ge\ \tfrac{r_0}{2}|a_t-A_t|^2-\Big(C_2+\tfrac{C_1^2}{2r_0}\Big)|y_t|^2 ,$$

the last step by the elementary inequality $2uv\le u^2+v^2$ with $u=\sqrt{r_0}\,|a_t-A_t|$ and $v=C_1|y_t|/\sqrt{r_0}$. The functions $t\mapsto|a_t-A_t|^2$ and $t\mapsto|y_t|^2$ are measurable and bounded (Step 1 and the composition lemma), so integrating this pointwise inequality over $[0,T]$ by monotonicity, inserting the terminal bound of (b) at $\Sigma=x_T$ into the identity of (a), and moving $\sum_\gamma P^\gamma_0y^\gamma_0$ to the left-hand side gives (c). $\blacksquare$