Beyond the Three Laws : Volume 2

# Beyond the Three Laws, Volume Two

### The Technical Practice

### Volume Two of Two, with a Living Companion on Open Problems

Volume One made the case: training a system's values to generalize is a stronger foundation for AI safety than constraining its outputs with rules. This volume is the honest accounting of how far current methods actually get you toward that — and exactly where they don't.

RLHF, Constitutional AI, red-teaming, interpretability, and the corrigibility training problem, explained in real technical terms for readers who already have ML background. No hand-waving, no overclaiming — every technique here is covered with its real, documented limitations alongside its real, documented strengths.

---

## What's inside

- What Pretraining Actually Gives You (and Doesn't) — the starting point every later technique has to work with

- RLHF — Teaching Through Preference — including a documented 2025 case study of exactly what happens when this goes wrong

- Constitutional AI and Self-Critique — how RLAIF reduces dependence on human labels without eliminating the underlying specification problem

- Red-Teaming — Breaking It on Purpose — and the honest limit of what a clean red-teaming pass can and can't prove

- Interpretability — Looking Inside — the most direct technical answer to Volume One's central open question

- The Corrigibility Training Problem, Specifically — why this commitment resists training more stubbornly than the other two

- Practitioner Objections, Addressed — DPO, larger rater pools, and whether any of this is actually testable

- A technical glossary — reward models, PPO, DPO, RLAIF, sparse autoencoders, the off-switch game

## Who this is for

Readers with working familiarity with how neural networks are trained — what a loss function is, roughly what generalization means. This is not an introduction to machine learning, and it assumes you've read Volume One first.

## Part of a two-volume project, plus a free living companion

Volume One makes the conceptual case with no ML background required. This volume covers the technical practice. Still Open, a free companion, tracks the open problems named across both volumes as the field's actual understanding evolves — updated after this volume, not frozen at whatever was true when it went to print.

## What you get

Instant download in PDF, EPUB. 16 pages.

---

Beyond the Three Laws, Volume Two — by Nathaniel Grey, published by Second Brain Publishing.