Keyboard Navigation
W
A
S
D
or arrow keys · M for map · Q to exit
← Back to exhibits
Data IntegrityAxiomAXM-007

Information Theory — Encoding, Entropy, and the Limits of Communication

Every encoding bug is a violation of Shannon's channel capacity theorem

Timeless · Mathematics · 12 min read

Claude Shannon published "A Mathematical Theory of Communication" in 1948. In 48 pages, he invented the field of information theory, defined the bit as a unit of information, proved that every communication channel has a maximum capacity, and showed that error-free communication is possible up to that capacity — and impossible beyond it. Every data corruption bug, every encoding failure, every truncation error, and every lossy compression artifact in the history of computing is a consequence of violating the theorems in that paper.

Shannon's insight was deceptively simple: information is the reduction of uncertainty. If you already know something, being told it again carries zero information. If there are 256 equally likely possibilities, identifying one of them requires exactly 8 bits of information (log₂(256) = 8). This is not a convention. It is a mathematical law. You cannot encode 256 possibilities in fewer than 8 bits without losing information.

Every time a developer truncates a field, chooses a character encoding, compresses an image, or decides how many digits to store in a date — they are making a claim about information capacity. When that claim is wrong, data is destroyed. Not corrupted. Not degraded. Destroyed — because the information content exceeded the channel capacity, and Shannon's theorem says the excess is irrecoverable.

Archaeologist's Note

This pattern has been found in applications built by talented developers at respected organizations across every decade of software history. Its presence in a codebase is not a reflection of the developer who wrote it — it is a reflection of what that developer was taught, what tools they had, and the path that was easiest given what they were taught. The goal is not to find fault. The goal is to find the pattern — before it finds you.

Katie's Law: The developers were not wrong. The shortcut was not wrong. The context changed and the shortcut didn't.

The AbstractionsInformation Theory2 / 5
Previous ExhibitMuseum MapNext Exhibit