Proof of The Regularised Entropic Rate Cost

Throughout we use two elementary inequalities. First, for all $s,t,c\in\mathbb{R}$,

Indeed, by symmetry we may assume $\max\{s,c\}\ge\max\{t,c\}$. If $\max\{s,c\}=c$ then $\max\{t,c\}\ge c=\max\{s,c\}$, so the two are equal and the left side is $0$. Otherwise $\max\{s,c\}=s$, and since $\max\{t,c\}\ge t$ we get $0\le\max\{s,c\}-\max\{t,c\}\le s-t\le|s-t|$. Second, for real numbers $s_1,s_2,s_3$ and $t_1,t_2,t_3$,

Indeed, writing $\varepsilon$ for the right-hand side, we have $s_i\ge t_i-\varepsilon\ge\min_j t_j-\varepsilon$ for each $i$, so $\min_i s_i\ge\min_j t_j-\varepsilon$; exchanging the roles of the two triples gives the reverse inequality.

Claim 1. The three entries of the minimum defining $\vartheta$ are the constant map $u\mapsto1$, which is Lipschitz with constant $0$; the map $u\mapsto\underline{a}^{-1}\max\{u,0\}$, which by the first inequality above is Lipschitz with constant $\underline{a}^{-1}$; and the map $u\mapsto\max\{0,2-u\bar{a}^{-1}\}$, which by the same inequality (applied with $c=0$ to the arguments $2-u\bar{a}^{-1}$ and $2-w\bar{a}^{-1}$) is Lipschitz with constant $\bar{a}^{-1}$. By the second inequality $\vartheta$ is therefore Lipschitz with constant $\max\{0,\underline{a}^{-1},\bar{a}^{-1}\}=\underline{a}^{-1}$, the last equality because $\underline{a}<1<\bar{a}$ gives $\bar{a}^{-1}<1<\underline{a}^{-1}$. Each of the three entries is nonnegative and the first equals $1$, so $0\le\vartheta\le1$ everywhere.

Write $M(u)=\max\{u,\underline{a}\}$, so that $M(u)\ge\underline{a}>0$ and $0<M(u)^{-1}\le\underline{a}^{-1}$ for every $u$. By the first inequality $|M(u)-M(w)|\le|u-w|$, whence

Since $\varpi=\vartheta\cdot M^{-1}$, for all $u,w$,

using $0\le\vartheta\le1$ and $0<M^{-1}\le\underline{a}^{-1}$. A Lipschitz map between metric spaces is continuous, so $\varpi$ is continuous. The bound $0\le\varpi\le\underline{a}^{-1}$ follows from $0\le\vartheta\le1$ and $0<M^{-1}\le\underline{a}^{-1}$.

For the values: if $u\le0$ then $\max\{u,0\}=0$, so the second entry of the minimum is $0$ and, all entries being nonnegative, $\vartheta(u)=0$ and $\varpi(u)=0$. If $u\ge2\bar{a}$ then $2-u\bar{a}^{-1}\le0$, so the third entry is $0$ and again $\vartheta(u)=\varpi(u)=0$. If $\underline{a}\le u\le\bar{a}$ then $\underline{a}^{-1}\max\{u,0\}=u\underline{a}^{-1}\ge1$ and $2-u\bar{a}^{-1}\ge2-1=1$, so all three entries are at least $1$ and the first equals $1$; hence $\vartheta(u)=1$, while $M(u)=u$, and $\varpi(u)=u^{-1}$. Finally, if $0<u<2\bar{a}$ then all three entries are strictly positive, so $\vartheta(u)>0$ and $\varpi(u)>0$.

Claim 2. Fix $u\in\mathbb{R}$ and choose real numbers $c<\min\{u,1\}$ and $d>\max\{u,1\}$. The function $\varpi$ is continuous on $[c,d]$ by claim 1, so by the first fundamental theorem of calculus on a closed interval the function $F(x)=\int_{[c,x]}\varpi(r)\,dr$ is defined for $x\in[c,d]$ and differentiable at every interior point $x$ of $[c,d]$ with $F'(x)=\varpi(x)$. By additivity of the integral over adjacent compact subintervals, $F(x)-F(1)=\int_{[1,x]}\varpi(r)\,dr$ for $1\le x\le d$ and $F(x)-F(1)=-\int_{[x,1]}\varpi(r)\,dr$ for $c\le x<1$; that is, $\Psi(x)=F(x)-F(1)$ for every $x\in[c,d]$. Since $u$ is interior to $[c,d]$, $\Psi$ is differentiable at $u$ with $\Psi'(u)=F'(u)=\varpi(u)$. As $u$ was arbitrary, $\Psi'=\varpi$ on $\mathbb{R}$.

For $w<u$ the same identity gives $\Psi(u)-\Psi(w)=\int_{[w,u]}\varpi(r)\,dr$, and $0\le\varpi\le\underline{a}^{-1}$ gives $0\le\Psi(u)-\Psi(w)\le\underline{a}^{-1}(u-w)$ by monotonicity of the integral; hence $\Psi$ is Lipschitz with constant $\underline{a}^{-1}$ and nondecreasing. From $\Psi(1)=0$ (a degenerate interval) and monotonicity, $\Psi\ge0$ on $[1,\infty)$ and $\Psi\le0$ on $(-\infty,1]$. Finally $\varpi$ vanishes off $[0,2\bar{a}]$ by claim 1, so for $u\ge1$ monotonicity and additivity give $0\le\Psi(u)=\int_{[1,\min\{u,2\bar{a}\}]}\varpi\le\int_{[0,2\bar{a}]}\varpi\le2\bar{a}\,\underline{a}^{-1}$, and for $u<1$ likewise $0\le-\Psi(u)=\int_{[\max\{u,0\},1]}\varpi\le2\bar{a}\,\underline{a}^{-1}$; so $|\Psi|\le2\bar{a}\,\underline{a}^{-1}$.

Claim 3. $\Psi$ is Lipschitz, hence continuous, so the argument of claim 2 applies verbatim with $\Psi$ in place of $\varpi$ and $\phi$ in place of $\Psi$: $\phi$ is differentiable at every $u\in\mathbb{R}$ with $\phi'(u)=\Psi(u)$. Consequently $\phi'=\Psi$ is differentiable at every point with $(\phi')'=\varpi$ by claim 2, and $\varpi$ is continuous by claim 1; so the first and second derivatives of $\phi$ exist and are continuous on $\mathbb{R}$, which by the correspondence between one-dimensional derivatives and partial derivatives on the real line is exactly the statement that $\phi$ is of class $C^{2}$ on $\mathbb{R}$ with $\phi''=\varpi$. The values $\phi(1)=0$ and $\phi'(1)=\Psi(1)=0$ are immediate from the degenerate integrals.

For uniqueness, let $\tilde{\phi}$ be of class $C^{2}$ on $\mathbb{R}$ with $\tilde{\phi}''=\varpi$, $\tilde{\phi}(1)=0$ and $\tilde{\phi}'(1)=0$. The function $\phi'-\tilde{\phi}'$ is continuous on $\mathbb{R}$ with vanishing derivative, so it is constant by A Continuous Function with Vanishing Derivative is Constant, and its value at $1$ is $0$; hence $\phi'=\tilde{\phi}'$. Then $\phi-\tilde{\phi}$ is continuous with vanishing derivative, hence constant by the same corollary, and vanishes at $1$; hence $\phi=\tilde{\phi}$.

Claim 4. For $u\ge1$ the integrand $\Psi$ is nonnegative on $[1,u]$ by claim 2, so $\phi(u)\ge0$ by monotonicity of the integral; for $u<1$ the integrand is nonpositive on $[u,1]$, so $\phi(u)=-\int_{[u,1]}\Psi\ge0$ as well.

Let $u>1$ and put $c_{1}=\min\{u,\bar{a}\}$, so $1<c_{1}\le\bar{a}$. For $r\in[1,c_{1}]$ we have $\varpi(s)=s^{-1}\ge\bar{a}^{-1}$ for $s\in[1,r]$ by claim 1, so $\Psi(r)=\int_{[1,r]}\varpi\ge\bar{a}^{-1}(r-1)$ by monotonicity. Since $\Psi\ge0$ on $[1,u]$, monotonicity and additivity give

Let $u<1$ and put $c_{2}=\max\{u,\underline{a}\}$, so $\underline{a}\le c_{2}<1$. For $r\in[c_{2},1]$ we have $\varpi(s)=s^{-1}\ge1$ for $s\in[r,1]$, so $-\Psi(r)=\int_{[r,1]}\varpi\ge1-r$. Since $-\Psi\ge0$ on $[u,1]$,

Together with $\phi(1)=0$ this shows that $\phi(u)>0$ for $u\neq1$ and that $1$ is the unique minimiser of $\phi$ on $\mathbb{R}$.

Since $\phi''=\varpi\ge0$, A Real Function with Nonnegative Second Derivative is Convex on an Interval shows that $\phi$ is convex on every compact interval $[c,d]\subset\mathbb{R}$; given $x,y\in\mathbb{R}$ and $\lambda\in[0,1]$, both $x,y$ and $\lambda x+(1-\lambda)y$ lie in the compact interval with endpoints $\min\{x,y\}$ and $\max\{x,y\}$, so the convexity inequality holds for them, and $\phi$ is convex on $\mathbb{R}$.

The remaining bounds are claims 1 and 2 restated through $\phi'=\Psi$ and $\phi''=\varpi$: $|\phi'|\le2\bar{a}\,\underline{a}^{-1}$ and $\phi'$ Lipschitz with constant $\underline{a}^{-1}$ by claim 2, and $0\le\phi''\le\underline{a}^{-1}$ with $\phi''$ Lipschitz with constant $2\,\underline{a}^{-2}$ by claim 1.