Entropy of Information
How much information is in a coin flip, a gene, a photograph, a language? This book explains the quantity that answers all of them: entropy of information, Claude Shannon's 1948 measure of uncertainty in bits, and the theory built on it. Chapters build the formula from its four exact readings — average surprise, binary questions, shortest description, missing knowledge — then follow it through coding (compression floors and capacity ceilings that run every zip file and satellite link), cryptography, the physics bridge to thermodynamics (Maxwell's demon, Landauer's price of erasure), quantum entanglement entropy, and machine learning, where cross-entropy now trains the largest artificial intelligence systems in existence. Later chapters cover entropy estimation as a scientific instrument, the field's honest open problems — semantic information, the interference channel, black holes — its landmark discoveries from Hartley to polar codes, and its future directions, ending with a measure invented for telephone wires that became the ledger of the information age.
Entropy of Information is part of the Aira Astralis Standalone collection by Aira Astralis: short, clear guides for anyone who looks up and wants to understand what they are seeing. DRM-free EPUB and PDF. Subtitle: The Physics of Data. Entropy of Information is part of Aira Astralis Standalone in the Aira Astralis library by Aira Astralis.