Proof of Completion of Squares and A Priori Control Bound for the Fluctuation Cost

Throughout we use the measurability conventions of Step 0 of the cost expansion theorem's proof: every integrand below becomes product-measurable after multiplication by the indicator of the regular event $\Omega_0$ (via the joint measurability lemma, the modified state-control map, and compositions with sequentially continuous functions), expectations are unaffected since $\Omega_0$ has probability $1$, and the Tonelli and Fubini theorems apply on the product of two finite measure spaces. Write $q_s=|\mathfrak{s}_s|^2+|\mathfrak{a}_s|^2$. Recall $|\mathfrak{s}_s|\le2\sqrt{N}$ everywhere and, by part (c) of the a priori second-moment bound with $\mathcal{A}<\infty$, $\sup_{t}\mathbb{E}[|\mathfrak{s}_t|^2]<\infty$, so $\int_{[0,T]}\mathbb{E}[q_s]\,ds<\infty$.

Part (a). Symmetry of $Q_t,R_t,\hat{F}$ is immediate from the symmetrized definitions. For the quadratic-form identities, split the sum over $i,j\in\{1,\dots,l+m\}$ into the four index blocks and use $z^iz^j=z^jz^i$: the state-state block contributes $\tfrac12\sum_{\gamma,\delta}H_{\gamma\delta}x^\gamma x^\delta=\sum_{\gamma,\delta}\tfrac14(H_{\gamma\delta}+H_{\delta\gamma})x^\gamma x^\delta=x\cdot Q_tx$; the two mixed blocks contribute $\tfrac12\sum_{\gamma,j}(H_{\gamma,l+j}+H_{l+j,\gamma})x^\gamma a^j=x\cdot V_ta$; the control-control block contributes $a\cdot R_ta$ in the same way; and the terminal identity is the state-state computation with $F$ in place of $H$. Under (H1), $R_t$ is symmetric with $a\cdot R_ta\ge r|a|^2>0$ for $a\neq0$, hence positive definite and invertible. Continuity in $t$: the entries of $E_t,\mathsf{B}_t$ are compositions of the continuous partials of $\bar{b}$ (part (i) of the drift regularity lemma) with the continuous trajectory pair; each $H_{ij}$ of the fluctuation linear-quadratic cost is continuous on $[0,T]$, being a composition of the continuous second partials of $\bar{L}$ (clause 2 of the cost extension) and of $\bar{b}$ (part (i) of the drift regularity lemma) with the continuous trajectory pair and the continuous co-state, so the entries of $Q_t,V_t,R_t$ are continuous; $t\mapsto R_t^{-1}$ has continuous entries by the continuity of the inverse of a continuous matrix function; and entries of $Z_t$ are continuous by (H2), so those of $W_t$ and $R_t^{-1}W_t^T$ are finite sums of products of continuous functions. Each entry is therefore bounded, so $C_Z$ and $C_K$ exist.

Part (b). Fix $(s,\omega)$ and $\delta\in\{1,\dots,l\}$. The segment from $(S_s,A_s)$ to $(\Sigma_s,\alpha_s)$ lies in $\Delta^l\times\mathbb{R}^m\subseteq U\times\mathbb{R}^m$: a convex combination of the simplex points $S_s$ and $\Sigma_s$ has nonnegative entries summing to $1$, and the control coordinates are unconstrained. Along it $|\partial_j\partial_i\bar{b}^\delta|\le3lK$ by part (iii) of the drift regularity lemma. Part (ii) of the Taylor expansion lemma (its second-order regularity hypothesis holding by part (i) of the drift regularity lemma), with $n=l+m$, $M_2=3lK$, and increment $w_s=N^{-1/2}(\mathfrak{s}_s,\mathfrak{a}_s)$, gives

using also the restriction clause of the drift regularity lemma to write $b$ for $\bar{b}$ on the simplex product. Multiplying by $\sqrt{N}$ and noting $\sqrt{N}\sum_i\partial_i\bar{b}^\delta(S_s,A_s)w^i_s=(E_s\mathfrak{s}_s+\mathsf{B}_s\mathfrak{a}_s)^\delta$ and $N|w_s|^2=q_s$, the $\delta$-th component of $e_s$ is bounded by $\tfrac32l(l+m)K\,N^{-1/2}q_s$; summing squares over the $l$ components gives $|e_s|\le\sqrt{l}\cdot\tfrac32l(l+m)K\,N^{-1/2}q_s=c_e\,N^{-1/2}q_s$.

Part (c). Apply part (b) of the weighted second-moment evolution lemma to the family $Z$ of (H2) at $t=T$, and use $Z_T=\hat{F}$:

By part (a) and the definition of the fluctuation linear-quadratic cost together with the Fubini theorem (each of the three groups is absolutely integrable: $|\mathbb{E}[\mathfrak{s}\cdot Q\mathfrak{s}]|$, $|\mathbb{E}[\mathfrak{s}\cdot V\mathfrak{a}]|$, $|\mathbb{E}[\mathfrak{a}\cdot R\mathfrak{a}]|$ are all bounded by constant multiples of $\mathbb{E}[q_s]$, whose time integral is finite),

Now substitute $g_s=E_s\mathfrak{s}_s+\mathsf{B}_s\mathfrak{a}_s+e_s$ and the Riccati density of (H2) into the integrand of $(\ast)$. Pointwise on $[0,T]\times\Omega$, using the symmetry of $Z_s$ (so that $2x\cdot Z(E x)=x\cdot(E^TZ+ZE)x$ for every $x$):

Adding the integrand $\mathfrak{s}\cdot Q\mathfrak{s}+\mathfrak{s}\cdot V\mathfrak{a}+\mathfrak{a}\cdot R\mathfrak{a}$ of $LQG$ and using $2\mathfrak{s}\cdot Z\mathsf{B}\mathfrak{a}+\mathfrak{s}\cdot V\mathfrak{a}=2\mathfrak{s}\cdot W\mathfrak{a}$,

the last step by expanding $u\cdot Ru$ with $u=\mathfrak{a}+R^{-1}W^T\mathfrak{s}$ and the symmetry of $R$ (the cross terms give $2\mathfrak{a}\cdot W^T\mathfrak{s}=2\mathfrak{s}\cdot W\mathfrak{a}$ and the square term gives $\mathfrak{s}\cdot WR^{-1}W^T\mathfrak{s}$, using $R R^{-1}=I$ and $(R^{-1})^T=R^{-1}$ for symmetric invertible $R$, from invertibility of symmetric positive definite matrices). Taking expectations and integrating in time (every displayed term is absolutely integrable: $\mathbb{E}[|u\cdot Ru|]$ is bounded by a constant multiple of $\mathbb{E}[q_s]$ since $|u_s|^2\le2|\mathfrak{a}_s|^2+2mlC_K^2|\mathfrak{s}_s|^2$ by the row-wise Cauchy-Schwarz estimate $|R^{-1}W^T\mathfrak{s}|^2\le mlC_K^2|\mathfrak{s}|^2$; and $|2\mathbb{E}[\mathfrak{s}\cdot Ze]|\le2lC_Z\mathbb{E}[|\mathfrak{s}||e|]\le4lC_Zc_e\,\mathbb{E}[q_s]$ by part (b) and $N^{-1/2}|\mathfrak{s}_s|\le2$), then adding the two displays and cancelling $\mathbb{E}[\mathfrak{s}_T\cdot\hat{F}\mathfrak{s}_T]$ yields the identity of (c).

Part (d). By (H1), pointwise $u_s\cdot R_su_s\ge r|u_s|^2$, so $r\int\mathbb{E}[|u_s|^2]ds\le\int\mathbb{E}[u_s\cdot R_su_s]ds$. Solve the identity of (c) for $\int\mathbb{E}[u_s\cdot R_su_s]ds$ and insert the expansion identity of part (c) of the cost expansion theorem, rewritten as $LQG[(\mathfrak{s}),(\mathfrak{a})]=N(J^N[h]-J^{MF})+\sum_\gamma P^\gamma_0\zeta^\gamma_N-R_N$. This gives

and the first display of (d) follows by bounding each subtracted term in absolute value: $|\sum_\gamma P^\gamma_0\zeta^\gamma_N|\le\sum_\gamma|P^\gamma_0||\zeta^\gamma_N|$; $|x\cdot Z_0x|\le C_Z(\sum_\gamma|x^\gamma|)^2\le lC_Z|x|^2$ (the elementary inequality $(\sum|x^\gamma|)^2\le l|x|^2$, as in the Taylor lemma's proof), so $|\mathbb{E}[\mathfrak{s}_0\cdot Z_0\mathfrak{s}_0]|\le lC_Z\mathbb{E}[|\mathfrak{s}_0|^2]$; $|\sum_{\gamma,\delta}Z^{\gamma\delta}_s\mathbb{E}[\Theta^{\gamma\delta}]|\le l^2C_Z\cdot2(l-1)B$ by the pathwise bound on $\Theta^{\gamma\delta}$ from part (a) of the martingale decomposition theorem; and $|2\mathbb{E}[\mathfrak{s}_s\cdot Z_se_s]|\le2lC_Z\mathbb{E}[|\mathfrak{s}_s||e_s|]\le2lC_Zc_eN^{-1/2}\mathbb{E}[|\mathfrak{s}_s|q_s]$ by part (b).

For the second display: $\mathfrak{a}_s=u_s-R_s^{-1}W_s^T\mathfrak{s}_s$, so $|\mathfrak{a}_s|^2\le2|u_s|^2+2|R_s^{-1}W_s^T\mathfrak{s}_s|^2\le2|u_s|^2+2mlC_K^2|\mathfrak{s}_s|^2$, where for each of the $m$ rows the Cauchy-Schwarz inequality gives $(\sum_\gamma(R^{-1}W^T)^{j\gamma}\mathfrak{s}^\gamma)^2\le lC_K^2|\mathfrak{s}|^2$; integrating proves $\mathcal{A}\le2\int\mathbb{E}[|u_s|^2]ds+2mlC_K^2\int\mathbb{E}[|\mathfrak{s}_s|^2]ds$.

For the third display: by part (ii) of the drift regularity lemma (as in the proof of the a priori second-moment bound), $\sum_\gamma(g^\gamma_s)^2\le\Lambda^2(|\mathfrak{s}_s|^2+|\mathfrak{a}_s|^2)\le\Lambda^2\big((1+2mlC_K^2)|\mathfrak{s}_s|^2+2|u_s|^2\big)\le\hat{\Lambda}^2\big(|\mathfrak{s}_s|^2+2|u_s|^2\big)$, using the bound on $|\mathfrak{a}_s|^2$ just derived and $\hat{\Lambda}^2=\Lambda^2(1+2mlC_K^2)\ge\Lambda^2$. Let $v(t)=\mathbb{E}[|\mathfrak{s}_t|^2]$, measurable and bounded by $4N$ (part (a) of the a priori second-moment bound), and $\mathcal{U}=\int_{[0,T]}\mathbb{E}[|u_s|^2]ds$, finite by the second display, $\mathcal{A}<\infty$, and $\sup_t v<\infty$; $s\mapsto\mathbb{E}[|u_s|^2]$ is measurable by the Tonelli theorem, the map $u$ being, after multiplication by the indicator of $\Omega_0$, a continuous-matrix combination of the product-measurable $\mathfrak{a}$ and $\mathfrak{s}$. Part (b) of the a priori second-moment bound (the three-term estimate) together with the display above gives, for every $t\in[0,T]$,

Define $w(t)$ as the bracketed constant plus $3T\hat{\Lambda}^2\int_{[0,t]}v(s)ds$; then $v\le w$, $w$ is continuous by the absolute continuity of the Lebesgue integral, $w(t)\le w(0)+3T\hat{\Lambda}^2\int_{[0,t]}w(s)ds$ by the monotonicity of the integral, the Lebesgue and Riemann integrals of the continuous $w$ agree, and Gronwall's lemma gives $w(t)\le w(0)\exp(3T\hat{\Lambda}^2t)$, hence the stated exponential bound for $v(t)$.