A/B Testing for Data Science Interviews — From Core Statistics to Product Experiment Decisions

A/B testing is the skill that separates a good data scientist from a great one in product company interviews. It's not just about knowing p-values — it's about showing you can connect statistical rigor to a real business decision.

This guide covers everything the interviewer is actually testing:

✅ The full A/B testing framework — in the exact order to follow in every interview answer

✅ Core concepts — alpha, beta, power, MDE, confidence intervals, effect size, sample size

✅ How to choose the right randomization unit and why it matters

✅ How to define primary metrics, guardrail metrics, and secondary metrics

✅ Stratified randomization — what it is and when to use it

✅ 2 fully worked experiments from real interview practice (incentive optimization + delivery marketplace)

✅ Goodhart's Law, novelty effect, primacy effect, peeking, multiple testing, spillover, SRM

✅ 4 model interview answers — including the p = 0.08 question and the "significant but tiny" question

✅ Pre-launch checklist

15 pages. Built from real practice, not theory alone.