Inference Machines: What Language Models Believe, and How They Change Their Minds

Can a language model weigh evidence the way Turing and Good did at Bletchley Park? This book measures it in their unit, the deciban: three decibans double the odds, twenty make them a hundred to one.

Chapter 1 plays Turing's imitation game with AI classifiers as judges and asks whether their probabilities mean what they say. The judges turn out to prefer answers like their own. Chapter 2 reads a model's chain of thought step by step and weighs how far each step moves the answer, like the lines of a credit scorecard. Chapter 3 treats fine-tuning as Bayes' rule, with the original model as the prior and the fine-tuned one as the posterior, and finds that fine-tuning overwrites the prior: told 32 times that the Eiffel Tower is in Rome, a model moves 93 decibans.

Two short essays frame the chapters: the familiar statistics inside a language model, from Markov to Bradley-Terry, and the unfamiliar engineering around it, from KV caches to agent harnesses. Line drawings throughout, interludes on Good's black crow and the Good-Turing estimator, and a glossary.

80 pages, PDF. Pay what you want, including nothing.