Entropy · Probability

Biased coin entropy

A coin's entropy H(p) = −p log₂ p − (1−p) log₂(1−p) peaks at exactly 1 bit per flip when p = ½ and falls to 0 as the coin becomes certain.

Builds on Surprise in bits. Taught in Bits & Surprise.

Drag P(heads) and watch a stream of flips: a fair coin is maximally unpredictable, a 95% coin produces a repetitive stream whose flips carry almost no information, and the readout follows the binary entropy curve.

The claim above is what this widget demonstrates. Disagree with it?

Where next