Proof of Regularity and Derivative Bounds of the Extended Aggregate State Drift

Throughout fix $\gamma\in\{1,\dots,l\}$, write $x=(\Sigma,\alpha)$ for points of $U\times V$, and recall from the extension definition its clauses 1-4, and from its preamble that $U$ and $V$ are open, convex and bounded, while $U\times V$ is an open subset of $\mathbb{R}^{l+m}$ as noted in its clause 2. We record that $U\times V$ is convex: a convex combination of two of its points has state block a convex combination of points of $U$ and control block one of points of $V$, and these lie in $U$ and $V$ respectively by their convexity. On $\Delta^l\times V$ we use, from the probability simplex: $\Sigma^\sigma\ge0$ for all $\sigma$ and $\sum_{\sigma=1}^{l}\Sigma^\sigma=1$; and on $\Delta^l\times\mathcal{A}$, from clause 1 together with clause 1 of the transition-rate family definition: $0\le\bar{\beta}(\sigma,\gamma',x)\le B$ for $x\in\Delta^l\times\mathcal{A}$ and all admissible index pairs.

(i). We first record a slice principle. Fix a point $a\in U\times V$; since $U\times V$ is open, there is a real $\rho>0$ such that every point of $\mathbb{R}^{l+m}$ at Euclidean distance less than $\rho$ from $a$ lies in $U\times V$. For $i\in\{1,\dots,l+m\}$ and $\tau\in\mathbb{R}$ write $a+\tau e_i$ for the point obtained by adding $\tau$ to the $i$th coordinate of $a$; its distance from $a$ is $|\tau|$ by claims 1 and 2 of the norm properties, so $a+\tau e_i\in U\times V$ for $\tau$ in the interval $J=(-\rho,\rho)$. For a function $g$ on $U\times V$ and the slice $G:J\to\mathbb{R}$, $G(\tau)=g(a+\tau e_i)$, the partial derivative of $g$ with respect to the $i$th variable exists at $a$ with value $L$ if and only if $G$ is differentiable at $0$ with $G'(0)=L$: both are the same $\varepsilon$-$\delta$ condition on the quotients $(g(a+\tau e_i)-g(a))/\tau$, the membership requirement in the partial-derivative definition holding for $0<|\tau|<\rho$ by the choice of $\rho$.

For $\sigma\in\{1,\dots,l\}$ let $\pi_\sigma:U\times V\to\mathbb{R}$ be the coordinate function $\pi_\sigma(x)=\Sigma^\sigma$. Its slice difference quotients of the preceding paragraph are constantly $\delta_{i\sigma}$, so $\partial_i\pi_\sigma$ exists at every point and equals the constant $\delta_{i\sigma}$; moreover $\pi_\sigma$ is continuous at every point, since $|\pi_\sigma(x)-\pi_\sigma(y)|\le d(x,y)$ by claims 2 and 4 of the norm properties, as are the constant functions; hence $\pi_\sigma$ and the constants are of class $C^1$ on $U\times V$ (clauses 1 and 3 of that definition). By the definition of the extended aggregate state drift,

Fix $i$ and a point $a\in U\times V$, with $J=(-\rho,\rho)$ as in the slice principle. Each function $\bar{\beta}(\sigma,\gamma',\cdot,\cdot)$ is of class $C^2$, hence of class $C^1$, on $U\times V$ by clause 2 of the extension definition and clause 2 of the $C^k$ definition; so by claim 2 of the segment-derivative lemma, applied with $W=U\times V$, $x=a$ and $h=e_i$, its slice on $J$ is differentiable at every interior point $\tau_0$ of $J$ with derivative $\partial_i\bar{\beta}(\sigma,\gamma',a+\tau_0 e_i)$, and the same holds for $\pi_\sigma$ with derivative $\delta_{i\sigma}$. The one-dimensional sum, constant-multiple and product rules then give that the slice of $\bar{b}^\gamma$ is differentiable at $0$, with derivative the value at $a$ of the first displayed formula of the statement; by the slice principle, $\partial_i\bar{b}^\gamma$ exists at $a$ and equals that formula. Each factor appearing in that formula is continuous at every point of $U\times V$ in the sense of Euclidean continuity: $\bar{\beta}(\sigma,\gamma',\cdot,\cdot)$ and its first partial derivatives by clauses 1 and 2 of the $C^k$ definition (the family being of class $C^2$), and the coordinate functions and constants as recorded above. By claim 1 of the continuity agreement lemma each of these functions is continuous on $U\times V$ relative to $U\times V$ as a map into the real line, so their finite sums and products are continuous on $U\times V$ by claim 5 of continuity of sums and products, and claim 1 of the agreement lemma translates this back into Euclidean continuity at every point. Hence $\partial_i\bar{b}^\gamma$ is continuous for every $i$, and $\bar{b}^\gamma$ itself is continuous by the same two-way translation applied to its defining formula; so $\bar{b}^\gamma$ is of class $C^1$ on $U\times V$ (clause 1 of the $C^k$ definition). Applying the same slice argument to the first displayed formula - a finite sum of products of coordinate functions, constants, the functions $\bar{\beta}(\sigma,\gamma',\cdot,\cdot)$, and their first partials, all of class $C^1$ on $U\times V$ (the first partials by clause 2 of the extension definition with clause 2 of the $C^k$ definition) - gives that $\partial_j\partial_i\bar{b}^\gamma$ exists at every point and equals the second displayed formula (using $\partial_j\pi_\sigma=\delta_{j\sigma}$ and that the constants $\delta_{i\sigma}$ have vanishing derivatives), and this expression is continuous by the same reasoning, the second partials $\partial_j\partial_i\bar{\beta}$ being continuous by clauses 1 and 2 of the $C^k$ definition; hence each $\partial_i\bar{b}^\gamma$ is of class $C^1$ on $U\times V$, and $\bar{b}^\gamma$ is of class $C^2$ there by clause 2 of the $C^k$ definition. Finally, for $x\in\Delta^l\times\mathcal{A}$, clause 1 allows replacing every $\bar{\beta}$ by $\beta$ in the defining formula of $\bar{b}^\gamma(x)$, which then coincides with the defining formula of the aggregate state drift of $\beta$ at $x$; hence $\bar{b}$ agrees with it on $\Delta^l\times\mathcal{A}$.

(ii). Let $x\in\Delta^l\times\mathcal{A}$ and $i\in\{1,\dots,l+m\}$. Estimating the four groups of terms of the first displayed formula separately: $\sum_{\sigma\neq\gamma}\delta_{i\sigma}|\bar{\beta}(\sigma,\gamma,x)|\le B$ (at most one $\sigma$ equals $i$); $\sum_{\sigma\neq\gamma}\Sigma^\sigma|\partial_i\bar{\beta}(\sigma,\gamma,x)|\le K\sum_{\sigma\neq\gamma}\Sigma^\sigma\le K$ by clause 3; $\delta_{i\gamma}\sum_{\sigma\neq\gamma}|\bar{\beta}(\gamma,\sigma,x)|\le(l-1)B$; and $\Sigma^\gamma\sum_{\sigma\neq\gamma}|\partial_i\bar{\beta}(\gamma,\sigma,x)|\le(l-1)K$. Altogether $|\partial_i\bar{b}^\gamma(x)|\le lB+lK=l(B+K)$.

For the Lipschitz estimate assume $\mathcal{A}$ is convex and let $x,y\in\Delta^l\times\mathcal{A}$. The segment from $x$ to $y$ stays in $\Delta^l\times\mathcal{A}$: a convex combination of two points of the simplex has nonnegative entries summing to $1$, hence lies in the simplex, and a convex combination of two points of $\mathcal{A}$ lies in $\mathcal{A}$ by convexity. The segment therefore lies in the open set $U\times V$, the first-order bound just proved holds at each of its points, and part (i) of the Taylor expansion lemma, applied to the $C^1$ function $\bar{b}^\gamma$ with $n=l+m$ and $M_1=l(B+K)$, gives $|\bar{b}^\gamma(x)-\bar{b}^\gamma(y)|\le\sqrt{l+m}\,l(B+K)\,d(x,y)$.

(iii). For $x\in\Delta^l\times V$, estimating the six groups of the second displayed formula with clause 3 and $\sum_{\sigma\neq\gamma}\Sigma^\sigma\le1$, $\Sigma^\gamma\le1$: the terms with $\delta_{i\sigma}$ and $\delta_{j\sigma}$ contribute at most $K$ each; the terms $\Sigma^\sigma\partial_j\partial_i\bar{\beta}(\sigma,\gamma,x)$ contribute at most $K$ in total; the terms with $\delta_{i\gamma}$ and $\delta_{j\gamma}$ contribute at most $(l-1)K$ each; and the terms $\Sigma^\gamma\partial_j\partial_i\bar{\beta}(\gamma,\sigma,x)$ contribute at most $(l-1)K$ in total. Hence $|\partial_j\partial_i\bar{b}^\gamma(x)|\le3K+3(l-1)K=3lK$.

For the uniform continuity claim, let $\varepsilon>0$ and set $C^*=2\,l\,\sqrt{l+m}\,K+2\,l\,K$ (a convenient over-bound: the exact coefficient collected below is $2l\sqrt{l+m}\,K+2(l-1)K$, and we over-estimate $l-1$ by $l$). By clause 4, applied to each of the finitely many admissible index pairs and taking the minimum of the resulting thresholds, there is $\delta_1>0$ such that $|\partial_j\partial_i\bar{\beta}(\sigma,\gamma',x)-\partial_j\partial_i\bar{\beta}(\sigma,\gamma',y)|\le\varepsilon/(2l)$ for all admissible indices whenever $d(x,y)\le\delta_1$. Set $\delta=\delta_1$ if $C^*=0$ and $\delta=\min(\delta_1,\varepsilon/(2C^*))$ otherwise. Let $x,y\in\Delta^l\times V$ with $d(x,y)\le\delta$, and take the difference of the second displayed formula at $x$ and at $y$ term by term. For the first-derivative factors: each $\partial_j\bar{\beta}(\sigma,\gamma',\cdot,\cdot)$ is of class $C^1$ on $U\times V$ (clause 2 of the extension definition with clause 2 of the $C^k$ definition) with partial derivatives bounded by $K$ on $U\times V$ (clause 3), so part (i) of the Taylor expansion lemma along the segment from $x$ to $y$ (which lies in $U\times V$, this set being convex as recorded at the outset) gives $|\partial_j\bar{\beta}(\sigma,\gamma',x)-\partial_j\bar{\beta}(\sigma,\gamma',y)|\le\sqrt{l+m}\,K\,d(x,y)$. For the product terms, writing $y=(\Sigma',\alpha')$:

and similarly with $\gamma$ in place of $\sigma$. Summing all contributions: the $\delta_{i\sigma}$ and $\delta_{j\sigma}$ groups give at most $2\sqrt{l+m}\,K\,d(x,y)$; the $\delta_{i\gamma}$ and $\delta_{j\gamma}$ groups give at most $2(l-1)\sqrt{l+m}\,K\,d(x,y)$; the two product groups give at most $2(l-1)K\,d(x,y)$ from the first summands (using $|\Sigma^\sigma-\Sigma'^\sigma|\le d(x,y)$ for every $\sigma$) plus $\tfrac{\varepsilon}{2l}\big(\sum_{\sigma\neq\gamma}\Sigma'^\sigma+(l-1)\Sigma'^\gamma\big)\le\tfrac{\varepsilon}{2l}\cdot l=\tfrac{\varepsilon}{2}$ from the second summands. Altogether the difference is at most $C^*d(x,y)+\varepsilon/2\le\varepsilon/2+\varepsilon/2=\varepsilon$, uniformly over $i,j,\gamma$, as claimed.