During the telecommunication boom, Claude Shannon, in his seminal 1948 paper¹, posed a question that would revolutionise technology:
How can we quantify communication?
Shannon’s findings remain fundamental to expressing information quantification, storage, and communication. These insights made major contributions to the creation of technologies ranging from signal processing, data compression (e.g., Zip files and compact discs) to the Internet and artificial intelligence. More broadly, his work has significantly impacted diverse fields such as neurobiology, statistical physics and computer science (e.g, cybersecurity, cloud computing, and machine learning).
[Shannon’s paper is the]
Magna Carta of the Information Age
— Scientific American
This is the first article in a series that explores information quantification — an essential tool for data scientists. Its applications range from enhancing statistical analyses to serving as a go-to decision heuristic in cutting-edge machine learning algorithms.