Proof of Taylor Expansion with Third-Order Remainder Bound

We treat $h\ge 0$; the case $h<0$ follows by the same argument on the interval $[x+h,x]$ with the roles of the endpoints exchanged, and $h=0$ is trivial.

Since $f$ is a $C^3$ map, the derivatives $f'$, $f''$, $f'''$ exist and are continuous on $\mathbb{R}$, so each is continuous on every closed interval and Riemann integrable there, and each of $f$, $f'$, $f''$ is an antiderivative of the next.

Step 1. For $0\le s$, by Fundamental Theorem of Calculus, Part II in One Dimension applied to $f''$ on $[x,x+s]$ and monotonicity of the Riemann integral (from upper and lower sums, an integrand bounded by $M_3$ in absolute value has integral bounded by $M_3 s$),

Step 2. For $0\le t$, by Fundamental Theorem of Calculus, Part II in One Dimension applied to $f'$ on $[x,x+t]$,

since the constant $f''(x)$ integrates to $f''(x)t$ (Fundamental Theorem of Calculus, Part II in One Dimension with the antiderivative $r\mapsto f''(x)\,r$), using linearity of the one-dimensional Riemann integral in the integrand (a basic consequence of its definition via Riemann sums, as in the proof of Independence of the Manifold Integral from Chart and Partition Choices). By Step 1 and monotonicity, with the antiderivative $r\mapsto M_3(r-x)^{2}/2$ of $r\mapsto M_3(r-x)$,

Step 3. By Fundamental Theorem of Calculus, Part II in One Dimension applied to $f$ on $[x,x+h]$, and again splitting off the polynomial part with explicit antiderivatives ($t\mapsto f'(x)\,t$ and $t\mapsto f''(x)\,t^{2}/2$),

By Step 2 and monotonicity, with the antiderivative $t\mapsto M_3(t-x)^{3}/6$ of $t\mapsto M_3(t-x)^{2}/2$,