Bits & Surprise
Shannon’s big idea, hands-on: measure surprise, sculpt entropy, build a working compressor, shout through noise, then KL divergence, mutual information, and the cross-entropy loss that trains modern AI. Check-ins along the way, graded final quiz at the end.
- Information is surprise
- Entropy: average information
- The shape of uncertainty
- A bit is a yes/no question
- The limit of compression
- Noisy channels and capacity
- KL divergence: model mismatch
- Mutual information
- Cross-entropy: the ML loss
- final quiz
Waves & Buckets
How sound becomes numbers and comes back alive: play with waves, hear aliasing fold past Nyquist, crush a melody to 3 bits, watch a DAC resurrect the signal, and tally the bitrate bill. Sound on; most demos are audible.
- Sound, amplitude, and frequency
- Sampling
- Nyquist and aliasing
- Bit depth and quantization
- ADC → DAC: the round trip
- Bitrate
- final quiz
Sines & Spectra
Signal processing from first principles: signals add, harmonics build any waveform, correlation measures one frequency at a time, two templates defeat unknown phase, the DFT runs the measurement everywhere, filtering edits the result, and roots of unity collapse the cost into the FFT, demonstrated live.
- Superposition: signals add
- Amplitude, frequency, and phase
- The rotating-point picture
- Building waveforms from harmonics
- Correlation
- Orthogonality
- The phase problem, two templates
- The discrete Fourier transform
- Filtering in the frequency domain
- The cost of the direct DFT
- Roots of unity
- The fast Fourier transform
- final quiz
Concepts
The building blocks behind the courses, each explained in a few minutes: a focused read, an interactive demo, and a quick check to make sure it stuck.