TheoremBase

The Entropy of a Nonnegative Function with Respect to a Probability Measure

Defines the entropy of a nonnegative measurable function GG with respect to a probability measure μ\mu, when GG and Glog⁡GG\log G are integrable, as the integral of Glog⁡GG\log G minus ϕ\phi of the integral of GG, where ϕ(s)=slog⁡s\phi(s)=s\log s and ϕ(0)=0\phi(0)=0.

Statement

In the setting of Measure Spaces and the Lebesgue Integral: Standing Notation, let (Ω,F,μ)(\Omega,\mathcal{F},\mu) be a measure space with μ(Ω)=1\mu(\Omega)=1, and let ϕ:[0,∞)→R\phi:[0,\infty)\to\mathbb{R} be the function of The Function slog⁡ss\log s: Continuity, Young's Inequality and Lower Bounds, with the Elementary Bounds for the Exponential and the Logarithm, with ϕ(0)=0\phi(0)=0 and ϕ(s)=slog⁡s\phi(s)=s\log s for positive ss.

(Entropy of a function) Let G:Ω→RG:\Omega\to\mathbb{R} be measurable with 0≤G(ω)0\le G(\omega) for every ω∈Ω\omega\in\Omega, such that GG and ϕ∘G\phi\circ G are integrable with respect to μ\mu; ϕ∘G\phi\circ G is measurable by The Function slog⁡ss\log s: Continuity, Young's Inequality and Lower Bounds, with the Elementary Bounds for the Exponential and the Logarithm §continuous, and ∫ΩG dμ≥0\int_{\Omega}G\,d\mu\ge0 by the monotonicity of Linearity and Monotonicity of the Lebesgue Integral §nonnegative. The entropy of GG with respect to μ\mu is the real number

Ent⁡μ(G)=∫Ωϕ∘G dμ−ϕ(∫ΩG dμ).\operatorname{Ent}_{\mu}(G)=\int_{\Omega}\phi\circ G\,d\mu-\phi\Bigl(\int_{\Omega}G\,d\mu\Bigr).

Citations

Loading…

Dependencies

Loading…

Related

0 relations

Curated associations between results. These are editable and subjective — they do not replace the dependency graph, which is derived from the references in the text.

No relations recorded yet.

Comments

Log in to comment.

Loading…