Proof of A Bound on the Second-Order Taylor Remainder by the Second and Third Derivatives

Fix $v,h\in\mathbb{R}$ and let $R=f(v+h)-f(v)-f'(v)h-\tfrac12f''(v)h^{2}$. If $h=0$, then $R=0$ and there is nothing to prove; let $h\neq0$, and put $w=v+h$, so that $w-v=h\neq0$.

The remainder as an increment of $f''$. Let $K=\bigl(f(w)-f(v)-f'(v)h\bigr)/h^{2}$, so that $R=Kh^{2}-\tfrac12f''(v)h^{2}$, and let $\psi:\mathbb{R}\to\mathbb{R}$ be the function

$$\psi(s)=f(s)+f'(s)(w-s)+K(w-s)^{2}.$$

Then $\psi(v)=f(v)+f'(v)h+Kh^{2}=f(w)$ by the choice of $K$, and $\psi(w)=f(w)$. By the sum and product rules, the function $\psi$ is differentiable, with

$$\psi'(s)=f'(s)+f''(s)(w-s)-f'(s)-2K(w-s)=(w-s)\bigl(f''(s)-2K\bigr)\qquad(s\in\mathbb{R})$$

(Derivatives of Constants, the Identity, Sums, Scalar Multiples, Products, Reciprocals, Quotients and Powers (i), Derivatives of Constants, the Identity, Sums, Scalar Multiples, Products, Reciprocals, Quotients and Powers (ii), Derivatives of Constants, the Identity, Sums, Scalar Multiples, Products, Reciprocals, Quotients and Powers (iv), Derivatives of Constants, the Identity, Sums, Scalar Multiples, Products, Reciprocals, Quotients and Powers (v), Derivatives of Constants, the Identity, Sums, Scalar Multiples, Products, Reciprocals, Quotients and Powers (vi)), and hence continuous (The Increment Form of the Derivative, and Differentiability Implies Continuity (ii)). Let $x=\min\{v,w\}$ and $y=\max\{v,w\}$, so $x<y$. By the mean value theorem on the interval $\mathbb{R}$ (Mean Value Theorem on a Closed Real Interval and on Any Interval (ii)) there is $\xi$ with $x<\xi<y$, that is, $\xi$ strictly between $v$ and $w$, and

$$0=\psi(y)-\psi(x)=\psi'(\xi)\,(y-x).$$

Since $y-x=|h|\neq0$ and $w-\xi\neq0$, this gives $f''(\xi)=2K$, and therefore

$$f(w)-f(v)-f'(v)h-\tfrac12f''(v)h^{2}=R=\tfrac12\bigl(f''(\xi)-f''(v)\bigr)h^{2}.$$

Two bounds on the increment. First, $|f''(\xi)-f''(v)|\le|f''(\xi)|+|f''(v)|\le2M_{2}$. Second, $\xi\neq v$ and $f''$ is differentiable, hence continuous (The Increment Form of the Derivative, and Differentiability Implies Continuity (ii)), so the mean value theorem (Mean Value Theorem on a Closed Real Interval and on Any Interval (ii)), applied to $f''$ on the interval $\mathbb{R}$ between $v$ and $\xi$, gives $\eta$ strictly between $v$ and $\xi$ with $f''(\xi)-f''(v)=f'''(\eta)\,(\xi-v)$; hence $|f''(\xi)-f''(v)|\le M_{3}|\xi-v|$. Since $\xi$ lies strictly between $v$ and $w$, $|\xi-v|<|w-v|=|h|$, and since $M_{3}\ge|f'''(v)|\ge0$, it follows that $M_{3}|\xi-v|\le M_{3}|h|$. Together,

$$|f''(\xi)-f''(v)|\le\min\bigl\{2M_{2},M_{3}|\xi-v|\bigr\}\le\min\bigl\{2M_{2},M_{3}|h|\bigr\},$$

and so

$$|R|\le\tfrac12\min\bigl\{2M_{2},M_{3}|h|\bigr\}h^{2}=\min\bigl\{M_{2}h^{2},\tfrac12M_{3}|h|^{3}\bigr\},$$

which is (i).