Proof of Twice Differentiability of a Sum in Separated Variables

Throughout we use the notation of the statement. We use repeatedly claim 3 of Concatenation Identifies a Product of Euclidean Spaces with a Euclidean Space, which gives

$$\iota(a_{1},a_{2})\cdot\iota(b_{1},b_{2})=a_{1}\cdot b_{1}+a_{2}\cdot b_{2},\qquad \lVert\iota(b_{1},b_{2})\rVert^{2}=\lVert b_{1}\rVert^{2}+\lVert b_{2}\rVert^{2},$$

and claim 2 of that lemma, which gives $\iota(a_{1},a_{2})+\iota(b_{1},b_{2})=\iota(a_{1}+b_{1},a_{2}+b_{2})$ and the corresponding identities for differences and scalar multiples. In particular, if $h=\iota(h_{1},h_{2})$ then $\lVert h_{1}\rVert^{2}\le\lVert h\rVert^{2}$ and $\lVert h_{2}\rVert^{2}\le\lVert h\rVert^{2}$, since the omitted summand is nonnegative, and hence $\lVert h_{1}\rVert\le\lVert h\rVert$ and $\lVert h_{2}\rVert\le\lVert h\rVert$ by claim 2 of Monotonicity of Squaring on the Nonnegative Elements of an Ordered Field.

Claim 1. Let $x\in U$ and write $x=\iota(\xi,\eta)$ with $\xi\in U_{1}$ and $\eta\in U_{2}$. Since $U_{1}$ and $U_{2}$ are open there are positive $\rho_{1},\rho_{2}$ with $\{\xi':d_{E}(\xi',\xi)<\rho_{1}\}\subseteq U_{1}$ and $\{\eta':d_{E}(\eta',\eta)<\rho_{2}\}\subseteq U_{2}$; let $\rho$ be the smaller of the two, which is positive. Let $x'\in\mathbb{R}^{N}$ with $d_{E}(x',x)<\rho$ and write $x'=\iota(\xi',\eta')$, which is possible in exactly one way since $\iota$ is a bijection. By claim 2 of Properties of the Euclidean Norm on $\mathbb{R}^n$: Square, Distance, Vanishing, Coordinate Bounds, Homogeneity, Triangle Inequalities and Expansion of a Square and the identities above, $d_{E}(x',x)=\lVert x'-x\rVert=\lVert\iota(\xi'-\xi,\eta'-\eta)\rVert$, so $\lVert\xi'-\xi\rVert\le\lVert x'-x\rVert$ and $\lVert\eta'-\eta\rVert\le\lVert x'-x\rVert$. Hence $d_{E}(\xi',\xi)<\rho_{1}$ and $d_{E}(\eta',\eta)<\rho_{2}$, so $\xi'\in U_{1}$, $\eta'\in U_{2}$ and $x'\in U$. Thus $U$ is open.

Claim 2. Let $\varepsilon\in\mathbb{R}$ be positive. By Twice Differentiability at a Point (i) there are positive $\delta_{1},\delta_{2}$ such that $\lVert h_{1}\rVert<\delta_{1}$ implies $\xi_{0}+h_{1}\in U_{1}$ and

$$\Bigl|v_{1}(\xi_{0}+h_{1})-v_{1}(\xi_{0})-p_{1}\cdot h_{1}-\tfrac{1}{2}h_{1}\cdot(X_{1}h_{1})\Bigr|\le\varepsilon\lVert h_{1}\rVert^{2},$$

and $\lVert h_{2}\rVert<\delta_{2}$ implies $\eta_{0}+h_{2}\in U_{2}$ and the corresponding estimate for $v_{2}$. Let $\delta$ be the smaller of $\delta_{1},\delta_{2}$. Let $h\in\mathbb{R}^{N}$ with $\lVert h\rVert<\delta$ and write $h=\iota(h_{1},h_{2})$; then $\lVert h_{1}\rVert<\delta_{1}$ and $\lVert h_{2}\rVert<\delta_{2}$ by the remark above, so $\xi_{0}+h_{1}\in U_{1}$ and $\eta_{0}+h_{2}\in U_{2}$, whence $x_{0}+h=\iota(\xi_{0}+h_{1},\eta_{0}+h_{2})\in U$ and

$$w(x_{0}+h)-w(x_{0})=\bigl(v_{1}(\xi_{0}+h_{1})-v_{1}(\xi_{0})\bigr)+\bigl(v_{2}(\eta_{0}+h_{2})-v_{2}(\eta_{0})\bigr).$$

Moreover $\iota(p_{1},p_{2})\cdot h=p_{1}\cdot h_{1}+p_{2}\cdot h_{2}$ and, by Block Diagonal Symmetric Matrices and the Blocks of a Symmetric Matrix (ii), $h\cdot\bigl((X_{1}\oplus X_{2})h\bigr)=h_{1}\cdot(X_{1}h_{1})+h_{2}\cdot(X_{2}h_{2})$. Therefore the quantity

$$w(x_{0}+h)-w(x_{0})-\iota(p_{1},p_{2})\cdot h-\tfrac{1}{2}h\cdot\bigl((X_{1}\oplus X_{2})h\bigr)$$

is the sum of the two quantities estimated above, so by claim 5 of Properties of the Absolute Value in an Ordered Field its absolute value is at most $\varepsilon\lVert h_{1}\rVert^{2}+\varepsilon\lVert h_{2}\rVert^{2}=\varepsilon\lVert h\rVert^{2}$. Since $X_{1}\oplus X_{2}\in\mathcal{S}(N)$ by Block Diagonal Symmetric Matrices and the Blocks of a Symmetric Matrix (i), this is the assertion.

Claim 3. Let $\varepsilon\in\mathbb{R}$ be positive and let $\delta$ be positive with the property that every $h\in\mathbb{R}^{N}$ with $\lVert h\rVert<\delta$ satisfies $x_{0}+h\in U$ and

$$\Bigl|w(x_{0}+h)-w(x_{0})-p\cdot h-\tfrac{1}{2}h\cdot(Zh)\Bigr|\le\varepsilon\lVert h\rVert^{2}. \tag{$\ast$}$$

First let $h_{1}\in\mathbb{R}^{m}$ with $\lVert h_{1}\rVert<\delta$ and put $h=\iota(h_{1},0_{\mathbb{R}^{n}})$, so that $\lVert h\rVert^{2}=\lVert h_{1}\rVert^{2}$ and hence $\lVert h\rVert=\lVert h_{1}\rVert<\delta$. Then $x_{0}+h=\iota(\xi_{0}+h_{1},\eta_{0})$ lies in $U$; since $\iota$ is injective and every element of $U$ is $\iota(\xi,\eta)$ with $\xi\in U_{1}$ and $\eta\in U_{2}$, this forces $\xi_{0}+h_{1}\in U_{1}$. Furthermore $w(x_{0}+h)-w(x_{0})=v_{1}(\xi_{0}+h_{1})-v_{1}(\xi_{0})$, and $p\cdot h=\iota(p_{1},p_{2})\cdot\iota(h_{1},0_{\mathbb{R}^{n}})=p_{1}\cdot h_{1}$, while the displayed quadratic form identity of Block Diagonal Symmetric Matrices and the Blocks of a Symmetric Matrix (v) gives

$$h\cdot(Zh)=h_{1}\cdot(Z^{11}h_{1})+2\,h_{1}\cdot(Z^{12}0_{\mathbb{R}^{n}})+0_{\mathbb{R}^{n}}\cdot(Z^{22}0_{\mathbb{R}^{n}})=h_{1}\cdot(Z^{11}h_{1}),$$

the last two terms vanishing because $Z^{12}0_{\mathbb{R}^{n}}$ and $Z^{22}0_{\mathbb{R}^{n}}$ are origins by claim 1 of Linearity, Compatibility with the Matrix Product, and a Norm Bound for the Matrix-Vector Product, and because the dot product of any point with an origin is $0$: an origin equals $0\cdot\zeta$ by Euclidean Space $\mathbb{R}^n$ is a Real Vector Space, so claim 5 of The Dot Product on $\mathbb{R}^n$ Is Symmetric, Bilinear and Positive Definite gives $w\cdot(0\cdot\zeta)=0\,(w\cdot\zeta)=0$. So $(\ast)$ reads

$$\Bigl|v_{1}(\xi_{0}+h_{1})-v_{1}(\xi_{0})-p_{1}\cdot h_{1}-\tfrac{1}{2}h_{1}\cdot(Z^{11}h_{1})\Bigr|\le\varepsilon\lVert h_{1}\rVert^{2}.$$

As $\varepsilon$ was arbitrary and $Z^{11}\in\mathcal{S}(m)$ by Block Diagonal Symmetric Matrices and the Blocks of a Symmetric Matrix (v), the function $v_{1}$ is twice differentiable at $\xi_{0}$ with first-order coefficient $p_{1}$ and Hessian $Z^{11}$. The same argument with $h=\iota(0_{\mathbb{R}^{m}},h_{2})$ shows that $v_{2}$ is twice differentiable at $\eta_{0}$ with first-order coefficient $p_{2}$ and Hessian $Z^{22}$.

Next we show that every entry of $Z^{12}$ is $0$. Fix $h_{1}\in\mathbb{R}^{m}$ and $h_{2}\in\mathbb{R}^{n}$ and put $g=\iota(h_{1},h_{2})$ and $A=h_{1}\cdot(Z^{12}h_{2})$. Let $\varepsilon$ be positive, let $\delta$ be as in $(\ast)$ for this $\varepsilon$, and let $\delta_{1},\delta_{2}$ be positive numbers witnessing, for the same $\varepsilon$, the twice differentiability of $v_{1}$ at $\xi_{0}$ and of $v_{2}$ at $\eta_{0}$ just established. Let $\tau$ be the smallest of $\delta$, $\delta_{1}$ and $\delta_{2}$, which is positive, and put $\Lambda=1+\lVert g\rVert$, which is positive and satisfies $\lVert g\rVert\le\Lambda$, $\lVert h_{1}\rVert\le\Lambda$ and $\lVert h_{2}\rVert\le\Lambda$ by the remark at the start of this proof. By claim 3 of The Archimedean Property of the Real Numbers there is a positive $\sigma$ with $\sigma<\tau\,\Lambda^{-1}$, and then $\sigma\Lambda<\tau$, so that $\sigma\lVert g\rVert<\delta$, $\sigma\lVert h_{1}\rVert<\delta_{1}$ and $\sigma\lVert h_{2}\rVert<\delta_{2}$. Put $h=\sigma g=\iota(\sigma h_{1},\sigma h_{2})$. Subtracting the two estimates for $v_{1}$ at increment $\sigma h_{1}$ and for $v_{2}$ at increment $\sigma h_{2}$ from $(\ast)$ at increment $h$, and using

$$w(x_{0}+h)-w(x_{0})=\bigl(v_{1}(\xi_{0}+\sigma h_{1})-v_{1}(\xi_{0})\bigr)+\bigl(v_{2}(\eta_{0}+\sigma h_{2})-v_{2}(\eta_{0})\bigr),$$

$p\cdot h=p_{1}\cdot(\sigma h_{1})+p_{2}\cdot(\sigma h_{2})$ and, by Block Diagonal Symmetric Matrices and the Blocks of a Symmetric Matrix (v),

$$h\cdot(Zh)=(\sigma h_{1})\cdot(Z^{11}\sigma h_{1})+2(\sigma h_{1})\cdot(Z^{12}\sigma h_{2})+(\sigma h_{2})\cdot(Z^{22}\sigma h_{2}),$$

we obtain, by claim 5 of Properties of the Absolute Value in an Ordered Field,

$$\bigl|\sigma^{2}A\bigr|\le\varepsilon\lVert h\rVert^{2}+\varepsilon\lVert\sigma h_{1}\rVert^{2}+\varepsilon\lVert\sigma h_{2}\rVert^{2}=2\varepsilon\sigma^{2}\lVert g\rVert^{2},$$

using $\lVert h\rVert^{2}=\sigma^{2}\lVert g\rVert^{2}$ and $\lVert\sigma h_{1}\rVert^{2}+\lVert\sigma h_{2}\rVert^{2}=\sigma^{2}\lVert g\rVert^{2}$. Dividing by the positive number $\sigma^{2}$ gives $|A|\le2\varepsilon\lVert g\rVert^{2}$. This holds for every positive $\varepsilon$, with $A$ and $\lVert g\rVert$ fixed; if $\lVert g\rVert^{2}=0$ then $|A|\le0$ and $A=0$ at once, while if $\lVert g\rVert^{2}$ is positive and $|A|$ were positive, then claim 3 of The Archimedean Property of the Real Numbers would supply a positive $\varepsilon$ with $\varepsilon<|A|\,\bigl(2\lVert g\rVert^{2}\bigr)^{-1}$, hence with $2\varepsilon\lVert g\rVert^{2}<|A|$, contradicting the bound just proved. Hence $A=0$; that is,

$$h_{1}\cdot(Z^{12}h_{2})=0\qquad\text{for all }h_{1}\in\mathbb{R}^{m},\ h_{2}\in\mathbb{R}^{n}.$$

Take $h_{1}=e_{i}$, the $i$th standard basis vector of $\mathbb{R}^{m}$, and $h_{2}=e_{j}$, the $j$th standard basis vector of $\mathbb{R}^{n}$. By Matrix-Vector Product the $i$th coordinate of $Z^{12}e_{j}$ is $\sum_{l=1}^{n}(Z^{12})_{il}(e_{j})_{l}=(Z^{12})_{ij}$, by claim 7 of Properties of Finite Sums; and the $i$th coordinate of a point $z$ equals $z\cdot e_{i}$ by Orthonormal Families, Standard Basis Vectors, and Plane Rotations of Euclidean Space, so $(Z^{12})_{ij}=e_{i}\cdot(Z^{12}e_{j})=0$. Thus every entry of $Z^{12}$ is $0$, and $Z=Z^{11}\oplus Z^{22}$ by Block Diagonal Symmetric Matrices and the Blocks of a Symmetric Matrix (v).