Two variables share I(X;Y) = H(X) + H(Y) − H(X,Y) bits: an umbrella that copies the weather perfectly shares all of the weather's entropy, one matched at reliability ½ shares none, and the overlap can never exceed the smaller of the two entropies.
Builds on Biased coin entropy, Binary symmetric channel. Taught in Bits & Surprise.
Set how often it rains and how reliably a citizen matches their umbrella to the sky, watch paired streams of weather and umbrellas, and read the two entropies as bars whose overlap is the mutual information and whose span is the joint entropy.
The claim above is what this widget demonstrates. Disagree with it?