Proof of Quadratic Expansion Bounds for the Mean-Field To-Go Value along a Stationary Mean-Field Triple

Throughout fix $t_{0}\in[0,T)$ and write $T^{\sharp}=T-t_{0}$; integrals over compact intervals are Lebesgue integrals as in the statement, and linearity and monotonicity of the integral are used freely. By claim 1 of the time-shift lemma, $(S^{\sharp},A^{\sharp},P^{\sharp})$ is a stationary mean-field triple for the shifted data (same $\beta$, $(L,G)$ and extensions, horizon $T^{\sharp}$), and $\sum_{\delta=1}^{l}|P^{\sharp\delta}_{t}|\le C_{P}$ for all $t\in[0,T^{\sharp}]$; so the number $C_{P}$ fixed in the statement is an admissible choice of the constant $C_{P}$ of the first-order expansion lemma for the shifted triple, and the constants $M_{2}$ and $C_{2}$ of that lemma for the shifted triple coincide with those of the statement. The control set $\mathcal{A}$ is nonempty, compact and convex as part of the data of the affine-controlled family, as recorded in the attainment theorem, so the expansion lemma applies to the shifted triple, its conclusion (b) included. Write $\mathcal{H}^{\sharp}_{t}(\Sigma,\alpha)$ and $\partial_{\gamma}\mathcal{H}^{\sharp}_{t}(\Sigma,\alpha)$ for the two-argument mean-field Hamiltonian and its state derivative coefficients of the expansion lemma for the shifted triple; this extends the one-argument $\mathcal{H}^{\sharp}_{t}$ adopted from the time-shift lemma, in the sense that $\mathcal{H}^{\sharp}_{t}(a)=\mathcal{H}^{\sharp}_{t}(S^{\sharp}_{t},a)$ for $a\in V$, as recorded in the expansion lemma.

Step 1: proof of conclusion 1. Fix $x\in\Delta^{l}$. By claim 3 of the time-shift lemma, $[A^{\sharp}]\in\mathcal{U}^{[T^{\sharp}]}_{\mathcal{A}}$ with admissible representative $A^{\sharp}$, and $S^{\sharp}=S^{[T^{\sharp}]}(S_{t_{0}},[A^{\sharp}])$. By the definition of the optimal value, $J^{*[T^{\sharp}]}_{x}$ is the infimum of the value set $\mathcal{V}^{[T^{\sharp}]}_{x}$ (notation of the time-shift lemma), hence a lower bound of it, so

$$J^{*[T^{\sharp}]}_{x}\le F^{[T^{\sharp}]}(x,[A^{\sharp}]).$$

Write $x^{\flat}=S^{[T^{\sharp}]}(x,[A^{\sharp}])$ for the flow of the horizon-$T^{\sharp}$ instance from $x$ under the same control, and $y_{t}=x^{\flat}_{t}-S^{\sharp}_{t}$ for $t\in[0,T^{\sharp}]$.

(1a) The pair $(x^{\flat},A^{\sharp})$ is an admissible pair for the shifted triple in the sense of the expansion lemma: by claim 2 of the flow stability lemma for the horizon-$T^{\sharp}$ instance and claim 1 of the existence and uniqueness theorem (initial value $x$, control $A^{\sharp}$), $x^{\flat}$ is continuous with values in $\Delta^{l}$ and satisfies $x^{\flat\gamma}_{t}=x^{\gamma}+\int_{[0,t]}\hat{b}^{\gamma}(x^{\flat}_{s},A^{\sharp}_{s})\,ds$ for all $t$ and $\gamma$; since $x^{\flat}_{s}\in\Delta^{l}$ and $A^{\sharp}_{s}\in\mathcal{A}$, claim 6 of the affine rate-family lemma gives $\hat{b}^{\gamma}(x^{\flat}_{s},A^{\sharp}_{s})=b^{\gamma}(x^{\flat}_{s},A^{\sharp}_{s})$, so this is the admissibility equation of the expansion lemma; and the components of $A^{\sharp}$ are continuous (claim 1 of the time-shift lemma), hence measurable by claim 3 of the toolkit.

(1b) Exact identity. Conclusion (a) of the expansion lemma, applied to the shifted triple and the pair $(x^{\flat},A^{\sharp})$ — whose control is the control of the triple — gives

$$J[(x^{\flat}),(A^{\sharp})]-J^{MF}[(S^{\sharp}),(A^{\sharp})]=\int_{[0,T^{\sharp}]}\mathrm{II}_{t}\,dt+\mathcal{G}-\sum_{\gamma=1}^{l}P^{\sharp\gamma}_{0}\,y^{\gamma}_{0},$$

where

$$\mathrm{II}_{t}=\mathcal{H}^{\sharp}_{t}(x^{\flat}_{t},A^{\sharp}_{t})-\mathcal{H}^{\sharp}_{t}(S^{\sharp}_{t},A^{\sharp}_{t})-\sum_{\gamma=1}^{l}\partial_{\gamma}\mathcal{H}^{\sharp}_{t}(S^{\sharp}_{t},A^{\sharp}_{t})\,y^{\gamma}_{t},\qquad \mathcal{G}=\bar{G}(x^{\flat}_{T^{\sharp}})-\bar{G}(S^{\sharp}_{T^{\sharp}})-\sum_{\gamma=1}^{l}\partial_{\gamma}\bar{G}(S^{\sharp}_{T^{\sharp}})\,y^{\gamma}_{T^{\sharp}},$$

and $J[(x^{\flat}),(A^{\sharp})]$ is the cost of the admissible pair defined there.

(1c) Pointwise bounds. Fix $t\in[0,T^{\sharp}]$. The segment from $(S^{\sharp}_{t},A^{\sharp}_{t})$ to $(x^{\flat}_{t},A^{\sharp}_{t})$ lies in $\Delta^{l}\times\mathcal{A}$ — for $\tau\in[0,1]$ the point $\tau x^{\flat}_{t}+(1-\tau)S^{\sharp}_{t}$ has nonnegative coordinates summing to $1$ — hence in the open sets $U_{c}\times\mathbb{R}^{m}$ and $U\times V$, because $\Delta^{l}\subset U_{c}$, $\Delta^{l}\subset U$ and $\mathcal{A}\subseteq V$ by the cost extension definitions. Its difference vector has components $y^{\gamma}_{t}$ in the first $l$ coordinates and $0$ in the last $m$, with Euclidean length $|y_{t}|$. Apply part (ii) of the multivariate Taylor lemma with $n=l+m$ along this segment to $f=\bar{L}$ on $U_{c}\times\mathbb{R}^{m}$, whose second-order partial derivatives are bounded by $K_{c}$ there (clause 3 of the cost extension definition), and to each $f=\bar{b}^{\delta}$ on $U\times V$, whose second-order partial derivatives are bounded by $3lK$ on $\Delta^{l}\times V$ (part (iii) of the regularity of the extended aggregate state drift), the segment lying in these sets as just shown. Multiplying the $\bar{b}^{\delta}$ estimates by $-P^{\sharp\delta}_{t}$, summing over $\delta$, adding the $\bar{L}$ estimate, and using $\sum_{\delta}|P^{\sharp\delta}_{t}|\le C_{P}$ and the definition of $\mathcal{H}^{\sharp}$ and $\partial_{\gamma}\mathcal{H}^{\sharp}$ in the expansion lemma:

$$|\mathrm{II}_{t}|\le\tfrac{1}{2}(l+m)\big(K_{c}+3\,l\,K\,C_{P}\big)\,|y_{t}|^{2}=C_{2}\,|y_{t}|^{2}.$$

For the terminal term, the second display of conclusion (b) of the expansion lemma for the shifted triple, at $\Sigma=x^{\flat}_{T^{\sharp}}\in\Delta^{l}$, gives $|\mathcal{G}|\le\tfrac{l\,K_{c}}{2}\,|y_{T^{\sharp}}|^{2}$.

(1d) Flow bound. Apply claim 4 of the flow stability lemma for the horizon-$T^{\sharp}$ instance with $(x_{0},\xi)=(S_{t_{0}},[A^{\sharp}])$ and $(x_{0}',\xi')=(x,[A^{\sharp}])$: the functionals of claim 3 there satisfy $g^{\gamma}_{r}(\xi')=\langle\xi'-\xi,w^{\gamma,r}\rangle_{L^{2}}=0$ for all $\gamma$ and $r$, since $\xi'=\xi$, so the constant that claim 4 requires as a bound for them (written $G$ there; in this proof that letter denotes the terminal cost everywhere else) may be taken to be $0$, and with $S^{[T^{\sharp}]}(S_{t_{0}},[A^{\sharp}])=S^{\sharp}$ (claim 3 of the time-shift lemma),

$$|y_{t}|\le e^{\Lambda_{b}T^{\sharp}}\,|x-S_{t_{0}}|\le e^{\Lambda_{b}T}\,|x-S_{t_{0}}|\qquad(t\in[0,T^{\sharp}]),$$

the second inequality because $T^{\sharp}\le T$, $\Lambda_{b}\ge0$ (by its definition in the affine rate-family lemma) and the real exponential function is strictly increasing by its basic properties.

(1e) Assembly. The map $t\mapsto|y_{t}|^{2}$ is continuous on $[0,T^{\sharp}]$ (its components being differences of continuous maps, and the squared norm a continuous function of them, by continuity of compositions and continuity of sums and products), hence measurable. Squaring the bound of (1d) gives $|y_{t}|^{2}\le e^{2\Lambda_{b}T}|x-S_{t_{0}}|^{2}$ for every $t\in[0,T^{\sharp}]$, using $e^{\Lambda_{b}T}e^{\Lambda_{b}T}=e^{2\Lambda_{b}T}$, the addition formula of the basic properties of the exponential. By monotonicity and claim 1 of the toolkit ($\lambda_{[0,T^{\sharp}]}([0,T^{\sharp}])=T^{\sharp}$),

$$\int_{[0,T^{\sharp}]}\mathrm{II}_{t}\,dt\le C_{2}\int_{[0,T^{\sharp}]}|y_{t}|^{2}\,dt\le C_{2}\,T^{\sharp}\,e^{2\Lambda_{b}T}\,|x-S_{t_{0}}|^{2}\le C_{2}\,T\,e^{2\Lambda_{b}T}\,|x-S_{t_{0}}|^{2},$$

the integrability of $\mathrm{II}$ being part of conclusion (a) of the expansion lemma. Combining (1b), (1c) and (1d), and using $P^{\sharp}_{0}=P_{t_{0}}$, $y_{0}=x-S_{t_{0}}$:

$$J[(x^{\flat}),(A^{\sharp})]\le J^{MF}[(S^{\sharp}),(A^{\sharp})]-\sum_{\gamma=1}^{l}P^{\gamma}_{t_{0}}\,\big(x^{\gamma}-S^{\gamma}_{t_{0}}\big)+\Big(C_{2}T+\tfrac{l\,K_{c}}{2}\Big)e^{2\Lambda_{b}T}\,\big|x-S_{t_{0}}\big|^{2}.$$

Now $J[(x^{\flat}),(A^{\sharp})]=\int_{[0,T^{\sharp}]}L(x^{\flat}_{t},A^{\sharp}_{t})\,dt+G(x^{\flat}_{T^{\sharp}})=F^{[T^{\sharp}]}(x,[A^{\sharp}])$: the first equality is the definition of the cost of an admissible pair in the expansion lemma, and the second holds because $F^{[T^{\sharp}]}(x,[A^{\sharp}])$ is, by the definition of the mean-field cost with the admissible representative $A^{\sharp}$, the generalized mean-field cost of the pair $(x^{\flat},A^{\sharp})$, which is the same Lebesgue integral plus terminal term. Also $J^{MF}[(S^{\sharp}),(A^{\sharp})]=\int_{[t_{0},T]}L(S_{t},A_{t})\,dt+G(S_{T})$ by claim 3 of the time-shift lemma. Chaining with $J^{*[T^{\sharp}]}_{x}\le F^{[T^{\sharp}]}(x,[A^{\sharp}])$ proves conclusion 1.

Step 2: proof of conclusion 2. Let $x\in\Delta^{l}$ with $|x-S_{t_{0}}|\le\varepsilon_{tg}$ and let $\xi\in\mathcal{U}^{[T^{\sharp}]}_{\mathcal{A}}$ ($t_{0}$ remaining the generic value fixed at the outset). Since $J^{*[T^{\sharp}]}_{x}$ is a lower bound of the value set $\mathcal{V}^{[T^{\sharp}]}_{x}$ and $F^{[T^{\sharp}]}(x,\xi)$ is a member of that set,

$$F^{[T^{\sharp}]}(x,\xi)\ \ge\ J^{*[T^{\sharp}]}_{x}.$$

By hypothesis (TG),

$$J^{*[T^{\sharp}]}_{x}\ \ge\ J^{*[T^{\sharp}]}_{S_{t_{0}}}-\sum_{\gamma=1}^{l}P^{\gamma}_{t_{0}}\,\big(x^{\gamma}-S^{\gamma}_{t_{0}}\big)-C_{tg}\,\big|x-S_{t_{0}}\big|^{2}.$$

By claim 5 of the time-shift lemma, applicable since $[A]\in\mathcal{M}^{*}_{S_{0}}$,

$$J^{*[T^{\sharp}]}_{S_{t_{0}}}=\int_{[t_{0},T]}L(S_{t},A_{t})\,dt+G(S_{T}).$$

Chaining the three displays gives conclusion 2. $\blacksquare$