Entropy · Probability

Surprise in bits

The information in a report of an event is −log₂ p bits, so each halving of the event's probability adds exactly one bit and a certain event carries none.

Builds on Doubling ruler. Taught in Bits & Surprise.

Slide the probability of an event, or pick a named event from a chip, and read off −log₂ p on the curve: a coin flip is 1 bit, a lottery win about 23.7 bits, the sun rising almost none.

The claim above is what this widget demonstrates. Disagree with it?

Where next