Proof of Lindeberg's Replacement Bound

Integrability. Let $k\in[n]$ and $X\in\{Y_{k},Z_{k}\}$. Then $g(X)=\min\{M_{2}X^{2},\tfrac12M_{3}a|X|^{3}\}$ is a random variable (here and below, the functions $y\mapsto M_{2}y^{2}$, $y\mapsto\tfrac12M_{3}a|y|^{3}$ and $y\mapsto|y|$ on $\mathbb{R}$ exist by Maps and Relations Given by Formulas (i), sums, products and Borel functions of measurable real functions are measurable, and finite sums in $\mathbb{R}$ are rearranged and compared termwise: Arithmetic of Measurable Real- and [0,∞]-Valued Maps (i), Arithmetic of Measurable Real- and [0,∞]-Valued Maps (ii), Measurable Maps: Generators, Composition, Continuous Maps, Restrictions, and Criteria for Real- and [0,∞]-Valued Maps (iv), Random Vectors: Coordinates, Functions, Pairs and Almost-Sure Equality (ii), Finite Sums in a Commutative Ring and in an Ordered Field: Distributivity, Differences, Telescoping, Constant Terms, Comparison and the Triangle Inequality (i), Finite Sums in a Commutative Ring and in an Ordered Field: Distributivity, Differences, Telescoping, Constant Terms, Comparison and the Triangle Inequality (v)), and $0\le g(X)\le M_{2}X^{2}$, since $M_{2}\ge|f''(0)|\ge0$ and likewise $M_{3}\ge0$. As $X\in\mathcal{L}^{2}(P)$, $X^{2}$ is integrable (Rules for Expectations, Variances and Covariances (vii)), so $g(X)$ is integrable (Linearity and Monotonicity of the Lebesgue Integral, and the Integrability Criterion (v)). This proves (i).

Interpolating sums. Let $U_{0}=a\sum_{k=1}^{n}Z_{k}$ and, recursively,

$$U_{j}=U_{j-1}+aY_{j}-aZ_{j}\qquad(j\in[n]),$$

and let $V_{j}=U_{j-1}-aZ_{j}$ for $j\in[n]$ ($j\mapsto U_{j}$ is a function on $\{0,1,\dots,n\}$ by Iterating a Binary Operation along a Finite List (i), applied to sums in $\mathbb{R}^{\Omega}$; $j\mapsto V_{j}$ is one on $[n]$, and $j\mapsto\mathbb{E}f(U_{j})$ one on $\{0,1,\dots,n\}$, by Maps and Relations Given by Formulas (i)). These are random variables, and for $j\in[n]$

$$U_{j}=V_{j}+aY_{j},\qquad U_{j-1}=V_{j}+aZ_{j}.$$

Finite sums of random variables are evaluated pointwise, $\bigl(a\sum_{k=1}^{n}Z_{k}\bigr)(\omega)=a\sum_{k=1}^{n}Z_{k}(\omega)$ for $\omega\in\Omega$, by the uniqueness in Iterating a Binary Operation along a Finite List (i). For fixed $j\in[n]$, let $m_{j}$ be the function on $[n]$ with $m_{j}(l)=aY_{l}$ for $l<j$, $m_{j}(j)=0$ and $m_{j}(l)=aZ_{l}$ for $l>j$ (Maps and Relations Given by Formulas (i)), and let $s_{j}=\sum_{l=1}^{j-1}aZ_{l}$ be the partial sum over $l<j$ (Iterated Operations: Finite Sums and Finite Products (iii), Iterated Operations: Finite Sums and Finite Products (ii)). Telescoping the recursion (Finite Sums in a Commutative Ring and in an Ordered Field: Distributivity, Differences, Telescoping, Constant Terms, Comparison and the Triangle Inequality (iii), Finite Sums in a Commutative Ring and in an Ordered Field: Distributivity, Differences, Telescoping, Constant Terms, Comparison and the Triangle Inequality (ii), Finite Sums in a Commutative Ring and in an Ordered Field: Distributivity, Differences, Telescoping, Constant Terms, Comparison and the Triangle Inequality (i)) gives $U_{j-1}=U_{0}-s_{j}+\sum_{l=1}^{j-1}aY_{l}$, and hence $V_{j}=U_{j-1}-aZ_{j}=\sum_{l=1}^{n}m_{j}(l)$; for $j=1$ the recursion is not used, since $V_{1}=U_{0}-aZ_{1}$ directly. Evaluated at each $\omega$, this is the closed form: for $j\in[n]$ and $\omega\in\Omega$,

$$V_{j}(\omega)=a\Bigl(\sum_{k=1}^{j-1}Y_{k}(\omega)+\sum_{k=j+1}^{n}Z_{k}(\omega)\Bigr),\qquad U_{n}=a\sum_{k=1}^{n}Y_{k},$$

empty sums being $0$.

The sum $V_{j}$ is independent of $Y_{j}$ and of $Z_{j}$. Fix $j\in[n]$, let $J_{j}=[2n]\setminus\{j,n+j\}$ and let $\mathcal{H}_{j}=\sigma\bigl(\bigcup_{i\in J_{j}}\sigma(W_{i})\bigr)$. The family $(\sigma(W_{i}))_{i\in[2n]}$ is independent (Independence of Events and of Random Elements (ii)), so by Independence of σ-Algebras: π-Systems, Grouping and Pairs (iii), applied to the disjoint index sets $J_{j}$ and $\{j\}$, and to $J_{j}$ and $\{n+j\}$, together with Independence of σ-Algebras: π-Systems, Grouping and Pairs (iv), $\mathcal{H}_{j}$ is independent of $\sigma(Y_{j})$ and of $\sigma(Z_{j})$ (the $\sigma$-algebra generated by $\sigma(W_{i})$ being $\sigma(W_{i})$ itself). Every $W_{i}$ with $i\in J_{j}$ is $\mathcal{H}_{j}$-measurable (The σ-Algebra Generated by Measurable Functions into Metric Spaces (ii)), hence so is $V_{j}$, by its closed form, and $\sigma(V_{j})\subseteq\mathcal{H}_{j}$ (The σ-Algebra Generated by Measurable Functions into Metric Spaces (i)). The product formula of Independence of Classes of Events and of σ-Algebras (iii) for $\mathcal{H}_{j}$ and $\sigma(Y_{j})$ holds in particular on $\sigma(V_{j})$, so $V_{j}$ and $Y_{j}$ are independent (Independence of Events and of Random Elements (iii)), and likewise $V_{j}$ and $Z_{j}$. Since $f'$ and $f''$ are differentiable, they are continuous (The Increment Form of the Derivative, and Differentiability Implies Continuity (ii), $\mathbb{R}$ being an interval by Basic Facts about Intervals of the Real Line (i)) and hence Borel, as is $y\mapsto y^{2}$; so, by Independent Random Elements: Preimages, Functions and Blocks (iv), for $X\in\{Y_{j},Z_{j}\}$

$$f'(V_{j})\text{ and }X\text{ are independent},\qquad f''(V_{j})\text{ and }X^{2}\text{ are independent}.$$

One replacement. For $v,h\in\mathbb{R}$ let

$$R(v,h)=f(v+h)-f(v)-f'(v)h-\tfrac12f''(v)h^{2}.$$

By A Bound on the Second-Order Taylor Remainder by the Second and Third Derivatives (i) and $a>0$, for all $v,y\in\mathbb{R}$,

$$|R(v,ay)|\le\min\bigl\{M_{2}a^{2}y^{2},\tfrac12M_{3}a^{3}|y|^{3}\bigr\}=a^{2}g(y).$$

Fix $j\in[n]$ and $X\in\{Y_{j},Z_{j}\}$, and write $V=V_{j}$. Pointwise on $\Omega$,

$$f(V+aX)=f(V)+a\,f'(V)X+\tfrac12a^{2}f''(V)X^{2}+R(V,aX).$$

The random variables $f(V+aX)$, $f(V)$, $f'(V)$ and $f''(V)$ are integrable, being Borel functions of random variables bounded by $M$ (Rules for Expectations, Variances and Covariances (ii), Linearity and Monotonicity of the Lebesgue Integral, and the Integrability Criterion (v); $f$ is continuous by The Increment Form of the Derivative, and Differentiability Implies Continuity (ii), $\mathbb{R}$ being an interval by Basic Facts about Intervals of the Real Line (i)). Moreover $X$ is integrable with $\mathbb{E}X=0$, and $X^{2}$ is integrable with $\mathbb{E}[X^{2}]=1$. By the independence just shown and Expectations and Variances of Independent Random Variables (i), $f'(V)X$ and $f''(V)X^{2}$ are integrable and

$$\mathbb{E}[f'(V)X]=\mathbb{E}f'(V)\,\mathbb{E}X=0,\qquad\mathbb{E}[f''(V)X^{2}]=\mathbb{E}f''(V)\,\mathbb{E}[X^{2}]=\mathbb{E}f''(V).$$

So $R(V,aX)$ is integrable, being a linear combination of the integrable terms above, and by linearity (Rules for Expectations, Variances and Covariances (iii))

$$\mathbb{E}f(V+aX)=\mathbb{E}f(V)+\tfrac12a^{2}\,\mathbb{E}f''(V)+\mathbb{E}R(V,aX).$$

Taking $X=Y_{j}$ and $X=Z_{j}$ and subtracting, the first two terms cancel:

$$\mathbb{E}f(U_{j})-\mathbb{E}f(U_{j-1})=\mathbb{E}R(V_{j},aY_{j})-\mathbb{E}R(V_{j},aZ_{j}).$$

Since $|R(V_{j},aX)|\le a^{2}g(X)$ pointwise, and $g(X)$ is integrable by (i),

$$|\mathbb{E}R(V_{j},aX)|\le\mathbb{E}|R(V_{j},aX)|\le a^{2}\,\mathbb{E}g(X)$$

(Rules for Expectations, Variances and Covariances (v), Rules for Expectations, Variances and Covariances (iv), Rules for Expectations, Variances and Covariances (iii)). Hence

$$\bigl|\mathbb{E}f(U_{j})-\mathbb{E}f(U_{j-1})\bigr|\le a^{2}\bigl(\mathbb{E}g(Y_{j})+\mathbb{E}g(Z_{j})\bigr)\qquad(j\in[n]).$$

Telescoping. Let $d_{j}=\mathbb{E}f(U_{j-1})$ for $j\in[n+1]$; this is a function on $[n+1]$, since $k\mapsto k+1$ maps $\{0,1,\dots,n\}$ onto $[n+1]$, and for $j\in[n]$ both $j$ and $j+1$ lie in $[n+1]$ (Intervals of Natural Numbers: Initial Segments, Adding One Element, Splitting and Shifting (iv), Intervals of Natural Numbers: Initial Segments, Adding One Element, Splitting and Shifting (ii)). By Finite Sums in a Commutative Ring and in an Ordered Field: Distributivity, Differences, Telescoping, Constant Terms, Comparison and the Triangle Inequality (iii),

$$\mathbb{E}f(U_{n})-\mathbb{E}f(U_{0})=d_{n+1}-d_{1}=\sum_{j=1}^{n}(d_{j+1}-d_{j})=\sum_{j=1}^{n}\bigl(\mathbb{E}f(U_{j})-\mathbb{E}f(U_{j-1})\bigr).$$

Each term of this sum lies between $-\delta_{j}$ and $\delta_{j}$, where $\delta_{j}=\bigl|\mathbb{E}f(U_{j})-\mathbb{E}f(U_{j-1})\bigr|$. Summing these bounds termwise, moving the sign through the sum and pulling out the factor $a^{2}$ (Finite Sums in a Commutative Ring and in an Ordered Field: Distributivity, Differences, Telescoping, Constant Terms, Comparison and the Triangle Inequality (v), Finite Sums in a Commutative Ring and in an Ordered Field: Distributivity, Differences, Telescoping, Constant Terms, Comparison and the Triangle Inequality (ii), Finite Sums in a Commutative Ring and in an Ordered Field: Distributivity, Differences, Telescoping, Constant Terms, Comparison and the Triangle Inequality (i)), and using the bound of the previous step,

$$\bigl|\mathbb{E}f(U_{n})-\mathbb{E}f(U_{0})\bigr|\le\sum_{j=1}^{n}\delta_{j}\le\sum_{j=1}^{n}a^{2}\bigl(\mathbb{E}g(Y_{j})+\mathbb{E}g(Z_{j})\bigr)=a^{2}\sum_{j=1}^{n}\bigl(\mathbb{E}g(Y_{j})+\mathbb{E}g(Z_{j})\bigr).$$

As $U_{n}=a\sum_{k=1}^{n}Y_{k}$ and $U_{0}=a\sum_{k=1}^{n}Z_{k}$, this is (ii).