Proof of Derivatives of a Function Along a Line Segment in ℝⁿ

Throughout, the arithmetic and order of $\mathbb{R}$ are used through The Structure of ℝ as a Field and an Ordered Ring (i), The Structure of ℝ as a Field and an Ordered Ring (ii), Arithmetic and Order Rules in the Real Numbers, Stated for ℝ (v), Arithmetic and Order Rules in the Real Numbers, Stated for ℝ (ix), Rules of Arithmetic and Order in an Ordered Field (iii) and The Quotient 1/x, Order Rules for Sums, and Nesting of Closed Intervals in ℝ (i); norms in $\mathbb{R}^{n}$ through Properties of the Euclidean Norm on $\mathbb{R}^n$: Square, Distance, Vanishing, Coordinate Bounds, Homogeneity, Triangle Inequalities and Expansion of a Square (vi) and Standard Basis Vectors of ℝ^q: Coordinates, Orthonormality and Unit Length (iii); and the metric spaces $(\mathbb{R}^{n},d_{E})$ and $(\mathbb{R},d_{\mathbb{R}})$ through $\mathbb{R}^n$ with the Euclidean Distance Is a Complete Metric Space with Coordinatewise Convergence (i) and The Absolute Value Metric on the Real Numbers (i).

We begin with (i). Taking $t=0$ and $t=1$ in Segments and Convex Subsets of $\mathbb{R}^n$ (i), the endpoints $x$ and $x+h$ lie in $[x,x+h]\subseteq U$, and for every $t\in[0,1]$ the point

$$x+th=(1-t)x+t(x+h)$$

lies on the segment, hence in $U$. Since $U$ is open, there are $\rho_{0},\rho_{1}>0$ with $B(x,\rho_{0})\subseteq U$ and $B(x+h,\rho_{1})\subseteq U$. If $h=0$, every $x+th$ equals $x$ and $r=1$ will do; otherwise put

$$r=\frac{\min(\rho_{0},\rho_{1})}{\lVert h\rVert}.$$

For $t\in(-r,0)$, the point $x+th$ is at distance $|t|\,\lVert h\rVert<\rho_{0}$ from $x$, and for $t\in(1,1+r)$ it is at distance $(t-1)\lVert h\rVert<\rho_{1}$ from $x+h$ (Properties of the Euclidean Norm on $\mathbb{R}^n$: Square, Distance, Vanishing, Coordinate Bounds, Homogeneity, Triangle Inequalities and Expansion of a Square (ii), Properties of the Euclidean Norm on $\mathbb{R}^n$: Square, Distance, Vanishing, Coordinate Bounds, Homogeneity, Triangle Inequalities and Expansion of a Square (v)); in both cases it lies in $U$. Together with the segment itself, this gives $x+th\in U$ for every $t\in(-r,1+r)$.

For the remaining clauses, let $\gamma:J\to\mathbb{R}^{n}$ be the function $\gamma(t)=x+th$. By hypothesis $\gamma(t)\in U$ for every $t\in J$, and $\varphi=f\circ\gamma$.

For (ii), let $f$ be differentiable on $U$. For $s\neq t$ in $J$ the difference quotient of $\gamma$ is constant,

$$\frac{\gamma(s)-\gamma(t)}{s-t}=h,$$

so $\gamma$ is differentiable at every $t\in J$ with $\gamma'(t)=h$, by The Derivative of a Vector-Valued Function as a Limit of Difference Quotients in Norm (i). Since $f$ is differentiable at $\gamma(t)$, the chain rule for a curve followed by a real-valued function, Chain Rule for Differentiable Functions Between Euclidean Spaces (iv), shows that $\varphi$ is differentiable at $t$ with

$$\varphi'(t)=\nabla f(x+th)\cdot h .$$

For (iii), let $f$ be of class $C^{1}$ on $U$. The curve $\gamma$ is continuous on $J$, since $\lVert\gamma(s)-\gamma(t)\rVert=|s-t|\,\lVert h\rVert$ (given $\varepsilon>0$, take $\delta=\varepsilon/(\lVert h\rVert+1)$; Continuous Map Between Metric Spaces (ii)). The function $f$, each partial derivative $\partial_{i}f$ and the gradient function $\nabla f$ are continuous on $U$, by Basic Properties of Cᵏ Functions (iv), by Basic Properties of Cᵏ Functions (iii) with $k=0$ and Basic Properties of Cᵏ Functions (i), and by Basic Properties of Cᵏ Functions (v), respectively. Composing with $\gamma$ (Continuity of Identity Maps, Compositions and Reciprocals (ii)), the functions $\varphi=f\circ\gamma$, $t\mapsto\partial_{i}f(x+th)$ and $t\mapsto\nabla f(x+th)$ are continuous on $J$. Since $f$ is differentiable on $U$ by Basic Properties of Cᵏ Functions (v), the derivative $\varphi'$ exists and is given by $\varphi'(t)=\nabla f(x+th)\cdot h$, as just shown; it is the dot product of the continuous function $t\mapsto\nabla f(x+th)$ with the constant vector $h$, hence continuous on $J$ by Continuity of Vector- and Matrix-Valued Maps: the Componentwise Criterion, Norms and Algebraic Operations (iv) and Continuity of Vector- and Matrix-Valued Maps: the Componentwise Criterion, Norms and Algebraic Operations (v).

For (iv), let $f$ be of class $C^{2}$ on $U$. For each $i\in[n]$, the partial derivative $\partial_{i}f$ is of class $C^{1}$ on $U$ by Basic Properties of Cᵏ Functions (iii), hence differentiable on $U$ by Basic Properties of Cᵏ Functions (v), so $\partial_{j}\partial_{i}f$ exists on $U$. The first-derivative formula, applied to $\partial_{i}f$ in place of $f$, shows that the function $\psi_{i}(t)=\partial_{i}f(x+th)$ is differentiable on $J$ with

$$\psi_{i}'(t)=\nabla(\partial_{i}f)(x+th)\cdot h=\sum_{j=1}^{n}\partial_{j}\partial_{i}f(x+th)\,h_{j},$$

by Gradient of a Real-Valued Function on an Open Subset of $\mathbb{R}^n$ (i) and the notation of Functions of Class Cᵏ: Notation (iii). The function $f$ itself is of class $C^{1}$ on $U$ by Basic Properties of Cᵏ Functions (ii), hence differentiable by Basic Properties of Cᵏ Functions (v), and the first-derivative formula expresses $\varphi'$ through the $\psi_{i}$:

$$\varphi'=\sum_{i=1}^{n}h_{i}\,\psi_{i}.$$

By the rules for derivatives of sums and constant multiples, applied $n$ times (Derivatives of Constants, the Identity, Sums, Scalar Multiples, Products, Reciprocals, Quotients and Powers (v) for each term $h_{i}\psi_{i}$, and Derivatives of Constants, the Identity, Sums, Scalar Multiples, Products, Reciprocals, Quotients and Powers (iv) in its $n$-term form Finite Sums of Real-Valued Maps: Continuity, Integral and Derivative (iii)), $\varphi'$ is therefore differentiable on $J$, and for every $t\in J$

$$\varphi''(t)=\sum_{i=1}^{n}\sum_{j=1}^{n}h_{i}h_{j}\,\partial_{j}\partial_{i}f(x+th)=\sum_{i=1}^{n}h_{i}\sum_{j=1}^{n}\bigl(D^{2}f(x+th)\bigr)_{ij}h_{j}=h\cdot\bigl(D^{2}f(x+th)\,h\bigr).$$

The middle equality comes from exchanging the names of the two summation indices and the entries of the Hessian matrix given by The Hessian Matrix of a Function of Class C² (i), with the order of summation exchanged by Iterated Operations over Finite Sets: Singletons, Disjoint Unions, Reindexing, Products of Sets, Termwise Combination, Homomorphisms and Intervals (iv) and the factor $h_{i}$ taken out of the inner sum by Finite Sums in a Commutative Ring and in an Ordered Field: Distributivity, Differences, Telescoping, Constant Terms, Comparison and the Triangle Inequality (i). The last equality is the definition of the dot product together with the entries of the matrix-vector product from Matrix-Vector Product (i),

$$\bigl(D^{2}f(x+th)\,h\bigr)_{i}=\sum_{j=1}^{n}\bigl(D^{2}f(x+th)\bigr)_{ij}h_{j}.$$

No symmetry of the Hessian is used.