Proof of Pair Expansion of the Weighted Likelihood-Ratio Square: Pair Covariances of Causal-Intensity Likelihoods, the Exact Expansion, and Its First-Order Form

Throughout, linearity refers to claim 2 of Linearity and Monotonicity of the Lebesgue Integral: finite sums and real scalar multiples of integrable functions are integrable, the integral of the combination is the combination of the integrals, and $|\int f\,d\rho|\le\int|f|\,d\rho$ for integrable $f$. Domination refers to the following consequence of claim 1 of that theorem (monotonicity for nonnegative functions) and of Integrable Function and the Lebesgue Integral (a measurable $f$ is integrable if and only if $\int|f|\,d\rho<\infty$): if $f$ is measurable and $|f|\le g$ pointwise with $g$ integrable, then $f$ is integrable and $|\int f\,d\rho|\le\int|f|\,d\rho\le\int g\,d\rho$.

Step 1 (claim 1). Fix $q,q'$. Claim 3 of Integer Power Products of Causal-Intensity Likelihoods: Pathwise Identity, Integral Bounds, and the Pair Identity, applied to the triple $(\mu,\mu_q,\mu_{q'})$ with the lower bound $\underline\mu$ for $\mu$, gives: $\ell>0$; $L_q$ and $L_{q'}$ are measurable; $\ell$, $\ell_q$, $\ell_{q'}$, $\ell_q\ell_{q'}/\ell$ and $\ell(1-L_q)(1-L_{q'})$ are integrable; $\tilde\mu_{qq'}$ is the causal intensity of claim 1 of that lemma for the exponents $(-1,1,1)$, whose exponent function is $E_{qq'}$ as displayed in the statement; and the pair identity $\int_{\mathbf{R}}\ell(1-L_q)(1-L_{q'})\,d\rho=\int_{\mathbf{R}}\frac{\ell_q\ell_{q'}}{\ell}\,d\rho-1 .$ Since $\ell L_qL_{q'}=\ell_q\ell_{q'}/\ell$ pointwise, this reads $\int\ell L_qL_{q'}\,d\rho=1+C_{qq'}$. The normalizations $\int\ell\,d\rho=1$ and $\int\ell L_q\,d\rho=\int\ell_q\,d\rho=1$ are claim 4 of Survival Identity, Measurability, Event-Count Recursion, and Normalization of the Likelihood of a Causal Intensity. By claim 1 of Integer Power Products of Causal-Intensity Likelihoods: Pathwise Identity, Integral Bounds, and the Pair Identity, $\tilde\mu_{qq'}$ is a causal intensity with bound $\bar\mu_q\bar\mu_{q'}/\underline\mu$, $E_{qq'}$ is measurable with $|E_{qq'}|\le\tilde{l}T(\bar\mu_q\bar\mu_{q'}/\underline\mu+\bar\mu+\bar\mu_q+\bar\mu_{q'})$, and $\ell_q\ell_{q'}/\ell=\ell_{\tilde\mu_{qq'}}\exp(E_{qq'})$ pointwise; by claim 4 of Survival Identity, Measurability, Event-Count Recursion, and Normalization of the Likelihood of a Causal Intensity applied to $\tilde\mu_{qq'}$, $\int\ell_{\tilde\mu_{qq'}}\,d\rho=1$, so $\ell_{\tilde\mu_{qq'}}$ is integrable, hence so is $\ell_{\tilde\mu_{qq'}}(\exp(E_{qq'})-1)=\ell_q\ell_{q'}/\ell-\ell_{\tilde\mu_{qq'}}$, with integral $(1+C_{qq'})-1=C_{qq'}$. The symmetry $C_{qq'}=C_{q'q}$ is immediate from the definition, and $\mathrm{Var}_q=C_{qq}=\int\ell(1-L_q)^{2}\,d\rho\ge0$ is finite as an integral of an integrable function; $\ell L_q=\ell_q$ is integrable.

Step 2 (claim 2). Pointwise on $\mathbf{R}$, expanding the square of the finite sum, $\ell\Bigl(\sum_{q}w_q(1-\mathsf{r}_qL_q)\Bigr)^{2}=\sum_{q,q'}w_qw_{q'}\,\ell(1-\mathsf{r}_qL_q)(1-\mathsf{r}_{q'}L_{q'}),\qquad \ell(1-\mathsf{r}_qL_q)(1-\mathsf{r}_{q'}L_{q'})=\ell-\mathsf{r}_q\ell L_q-\mathsf{r}_{q'}\ell L_{q'}+\mathsf{r}_q\mathsf{r}_{q'}\ell L_qL_{q'} .$ Each of the four functions on the right is integrable by Step 1, so every term, and hence the left side, is integrable, and by linearity and Step 1, $\int_{\mathbf{R}}\ell(1-\mathsf{r}_qL_q)(1-\mathsf{r}_{q'}L_{q'})\,d\rho=1-\mathsf{r}_q-\mathsf{r}_{q'}+\mathsf{r}_q\mathsf{r}_{q'}(1+C_{qq'})=(1-\mathsf{r}_q)(1-\mathsf{r}_{q'})+\mathsf{r}_q\mathsf{r}_{q'}C_{qq'} .$ Summing with the weights $w_qw_{q'}$ and using $\sum_{q,q'}w_qw_{q'}(1-\mathsf{r}_q)(1-\mathsf{r}_{q'})=(\sum_qw_q(1-\mathsf{r}_q))^{2}$ gives claim 2.

Step 3 (two inequalities for the exponential). For every real $x$, $|\exp(x)-1|\le\exp(|x|)-1\qquad\text{and}\qquad|\exp(x)-1-x|\le\tfrac12x^{2}\exp(|x|).$ Indeed, by The Real Exponential Function, $\exp(x)=\sum_{k\ge0}x^{k}/k!$ as a convergent series, so $\exp(x)-1=\sum_{k\ge1}x^{k}/k!$ and $\exp(x)-1-x=\sum_{k\ge2}x^{k}/k!$ (the partial sums differ from those of the series for $\exp(x)$ by the omitted initial terms, and limits respect this by Arithmetic of Limits of Real Sequences). The absolute value of a limit of partial sums is at most the limit of the partial sums of absolute values (triangle inequality for finite sums, continuity of the absolute value under limits by Arithmetic of Limits of Real Sequences, and claim 1 of Order Properties of Limits of Real Sequences), so $|\exp(x)-1|\le\sum_{k\ge1}|x|^{k}/k!=\exp(|x|)-1$ and $|\exp(x)-1-x|\le\sum_{k\ge2}|x|^{k}/k!$. For $k\ge2$, $k!\ge2\,(k-2)!$ (as $k(k-1)\ge2$), so $|x|^{k}/k!\le\tfrac12x^{2}|x|^{k-2}/(k-2)!$; comparing partial sums and passing to the limit (Order Properties of Limits of Real Sequences), $\sum_{k\ge2}|x|^{k}/k!\le\tfrac12x^{2}\sum_{j\ge0}|x|^{j}/j!=\tfrac12x^{2}\exp(|x|)$.

Step 4 (claim 3). Let $\bar{E}$ and $\mathsf{N}$ be as in claim 3, and write $\ell_{qq'}=\ell_{\tilde\mu_{qq'}}$ and $E=E_{qq'}$. Since $E$ is measurable and bounded and $\ell_{qq'}$ is integrable, $\ell_{qq'}E$ is integrable by domination. Let $M$ and $M'$ be bounds for $|\exp(E)-1|$ and $|\exp(E)-1-E|$ on $\mathbf{R}$ (they exist since $E$ is bounded). On $\mathbf{R}\setminus\mathsf{N}$, Step 3 and the monotonicity of $\exp$ (claim 4 of Basic Properties of the Exponential Function) give $|\exp(E)-1|\le\exp(\bar{E})-1$ and $|\exp(E)-1-E|\le\tfrac12\bar{E}^{2}\exp(\bar{E})$. First, $\int\ell_{qq'}\mathbf{1}_{\mathsf{N}}\,d\rho=0$: for every natural number $k$, $\min(\ell_{qq'},k)\mathbf{1}_{\mathsf{N}}\le k\mathbf{1}_{\mathsf{N}}$, so by monotonicity (claim 1 of Linearity and Monotonicity of the Lebesgue Integral) and the integral of a multiple of an indicator (Simple Function and Its Integral), $\int\min(\ell_{qq'},k)\mathbf{1}_{\mathsf{N}}\,d\rho\le k\rho(\mathsf{N})=0$ (the map $\min(\ell_{qq'},k)$ being measurable by claim 4 of Arithmetic, Absolute Values, and Pointwise Limits of Measurable Real-Valued Functions), and Monotone Convergence Theorem applies as $k\to\infty$. Put $g_1=(\exp(\bar E)-1)\ell_{qq'}\mathbf{1}_{\mathbf{R}\setminus\mathsf{N}}+M\ell_{qq'}\mathbf{1}_{\mathsf{N}}$; then $|\ell_{qq'}(\exp(E)-1)|\le g_1$ pointwise, $g_1\le\max(\exp(\bar E)-1,M)\,\ell_{qq'}$ is integrable, and by linearity $\int g_1\,d\rho=(\exp(\bar E)-1)\int\ell_{qq'}\mathbf{1}_{\mathbf{R}\setminus\mathsf{N}}\,d\rho+M\cdot0\le\exp(\bar E)-1$, using $\int\ell_{qq'}\mathbf{1}_{\mathbf{R}\setminus\mathsf{N}}\,d\rho\le\int\ell_{qq'}\,d\rho=1$. Hence, by Step 1 and domination, $|C_{qq'}|=|\int\ell_{qq'}(\exp(E)-1)\,d\rho|\le\int g_1\,d\rho\le\exp(\bar E)-1$. The same argument with $\ell_{qq'}(\exp(E)-1-E)$ and the majorant $g_2=\tfrac12\bar E^{2}\exp(\bar E)\ell_{qq'}\mathbf{1}_{\mathbf{R}\setminus\mathsf{N}}+M'\ell_{qq'}\mathbf{1}_{\mathsf{N}}$ gives $|\int\ell_{qq'}(\exp(E)-1-E)\,d\rho|\le\tfrac12\bar E^{2}\exp(\bar E)$, and by linearity $\int\ell_{qq'}(\exp(E)-1-E)\,d\rho=C_{qq'}-\int\ell_{qq'}E\,d\rho$, which is the second inequality. $\blacksquare$