RAG Anti-Patterns — Part 2: Prompt, Evaluation, Security, Cost & Organizational Sins — 42 Templates & Checklists for Auditing Prompt, Evaluation, Security, Cost & Organizational Failures in Production RAG

Retrieval was fine — and the answer still went wrong, leaked, or cost a fortune. 42 anti-pattern audit templates & checklists distilled from Chapters 7–12 and Appendices A & B of RAG Anti-Patterns: the four prompt sins, the four evaluation sins, the four security sins, the four cost sins, the four organizational sins, and the catalogue habit that audits all of it on a calendar — each card naming the sin, its detection signal, and the book's lasting fix. PDF + Markdown source.

## A new support agent asked for the escalation path. The assistant quoted an HR memo and a salary band.

Nothing had been breached from outside. The documents had simply been indexed together, with no permissions in the retrieval layer, and the assistant — helpful, fluent, and wrong about who was entitled to what — synthesized across all of it. **RAG Anti-Patterns — Part 2** is the working audit for the half of RAG failure that lives above and around the pipeline: 42 copy-paste audit templates and checklists distilled from Chapters 7–12 and Appendices A & B of the dedicated anti-patterns book — govern the prompt as a control surface, make evaluation honest enough to say no, treat retrieval as an authorization boundary, divide cost by correct answers instead of queries, name an owner for every stage, and turn the whole catalogue into a recurring audit habit with the book's own appendix checklist and quick reference.

Every card carries the book's tested positions built in: a prompt is fixed by subtraction and structure, never by one more emphatic line; citation coverage and citation precision are reported together, or the number is theatre; a format is a schema that can fail, be counted, and carry a decided consequence — not a stronger request; retrieved evidence outranks hard-coded prompt facts, and abstention is a designed behaviour, not a failure; sliced reporting with per-slice floors lets the smallest red bar veto the largest green one, and an LLM judge earns authority only by agreeing with accountable humans, measured per judge version; retrieval is an authorization boundary — filter before you rank, treat retrieved text as data never as instructions, log as if the log will be read by someone who should not see it, and bind every cached answer to the permission set that produced it; cost is divided by correct answers, per query class, context is a purchase whose tokens must earn their place, and spend is metered at the point of spend; retrieval quality needs a named owner with a pager, a gate, and the authority to say not yet; and every finding is scored blast radius times frequency, security boosted, fixed one architectural cause at a time, and pinned with a regression test so it cannot quietly return. Where the book gives a number — its verdict gates, its illustrative audit readings, its worked calculator outputs — the card carries the book's number and labels it as the book's or as illustrative, exactly as the book does. Bracketed fields are yours to fill; nothing here invents a threshold for your system.

### What's inside — 42 templates & checklists in 7 sections

**1. Instruction Soup — The Prompt Sins (PRM-1 to PRM-4) (6)** — PRM-1 Instruction Soup — Prompt Lint & Clause Inventory Card · Instruction Ablation Test Sheet — One Clause at a Time · PRM-2 The Optional Citation — Citation Coverage & Precision Audit Card · PRM-3 The Format Lottery — Schema-Validation Failure Log & Contract Card · PRM-4 The Fighting Prompt — Conflict Test & Abstention Measurement Card · Prompt Release Gate & Change-Control Checklist

**2. Vibes-Based Testing — The Evaluation Sins (EVL-1 to EVL-4) (6)** — EVL-1 The Demo Set — Evaluation Provenance Audit Card · Production-Distribution Evaluation Rebuild Sheet · EVL-2 The Pooled Average — Sliced Reporting & Per-Slice Floor Sheet · EVL-3 The Agreeable Judge — Judge Calibration & Bias-Probe Card · EVL-4 The Frozen Gold — Refresh, Contamination Scan & Divergence Watch Card · Sliced Reporter & Cohen's Kappa Run Sheet

**3. The Open Vault — The Security Sins (SEC-1 to SEC-4) (6)** — SEC-1 The ACL-Free Index (The Helpful Leak) — Permission-Parity Test Card · SEC-2 The Confused Deputy — Injection Canary & Tool-Provenance Card · SEC-3 The Log That Leaks — Log Content, Retention & Access Audit Card · SEC-4 The Borrowed Answer — Cross-Permission Cache Test Card · Permission-Parity Detector Run Sheet · Quarterly Security Audit Board

**4. The Meter Is Running — The Cost Sins (CST-1 to CST-4) (6)** — CST-1 The Flagship for Everything — Cost-per-Correct-Answer Audit Card · CST-2 The Full Re-Embed Habit — Embeddings-per-Changed-Document Audit Card · CST-3 The Context Bonfire — Cost-per-Cited-Token & Utilisation Audit Card · CST-4 The Unmetered Tenant — Attribution Coverage Audit Card · Routing Comparison Calculator Run Sheet · Sliced Cost Review Board

**5. Nobody's System — The Organizational Sins (ORG-1 to ORG-4) (6)** — ORG-1 Demo-Driven Development — Demo-vs-Production Divergence Audit Card · ORG-2 The Empty Chair — Ownership Registry & Detection-Time Audit Card · ORG-3 The Blame Relay — Postmortem Archaeology & Evidence-Standard Card · ORG-4 The Frozen Project — Post-Launch Audit & Product Conversion Card · Ownership Readiness Scorer Run Sheet · Organizational Audit Board

**6. The Catalogue Habit — Auditing Yourself Before Production Does (6)** — Standing Instrument Panel Sheet — One Signal per Family, Ten Families · Harm × Habit Scoring Sheet — Blast Radius × Frequency, Security Boosted · One Fix, Pinned — Remediation Record Card · Audit Cadence & Ownership Sheet — Monthly Light Pass, Quarterly Full Audit · Incident Graduation Worksheet — Every Incident Into the Catalogue · Instrument Baseline & Silence-Test Record

**7. Part 2 Gates & Working Artefacts — Chapters 7–12 + Appendices A & B (6)** — Part 2 Anti-Pattern Register — All Twenty Sins on One Board · Appendix A Quarterly Audit Checklist — the book's full-catalogue audit checklist carried line for line (38 lines) · Appendix B Quick-Reference Wall Card — Ten Families & the Three Rules · The Recurring Self-Check — the audit-habit capstone: a monthly light pass and quarterly full audit on one dated, owned grid · Audit Cycle Scoring & Remediation Log · Part 2 Production Gate — Chapters 7–12 Sign-Off — the book's six chapter checklists consolidated into one evidence-backed gate (31 lines)

### Who this is for

RAG platform and ML engineers who own a production retrieval system, evaluation and quality leads who sign its release reports, security-minded engineers responsible for what the assistant may reveal, and technical leads asked why a system that passed every gate still fails, leaks, or costs too much in production. The source book assumes you already run — or are about to run — a retrieval-augmented system; no research background is required. If nobody in your organisation can say what your full system prompt actually does, start at Template 1.

### What this pack is not

- **Not Part 1.** Ingest sins, chunking sins, embedding sins, index sins, and retrieval sins — Chapters 1–6 — live in the Part 1 companion pack; nothing here restates their audit cards, and this pack is complete and useful without them. Three artefacts in this pack (the Chapter 12 instrument panel and the book's Appendix A checklist and Appendix B quick reference) are the book's own whole-catalogue summaries: they name the Part 1 families by signature and standing detection signal only, exactly as the book prints them, and ship no Part 1 template.

- **No runnable code or tooling.** The book's citation checker, sliced reporter, permission-parity detector, routing calculator, and ownership scorer are described as audit procedures and run sheets inside the cards — the pack ships the templates that specify, run, and record these audits, not a working checker, monitor, or pipeline component.

- **No vendor or product guidance.** Models, parsers, vector stores, caches, and logging platforms are treated as patterns and measurement problems, exactly as the book treats them. No product is selected, installed, or configured here.

- **No benchmark, savings, or results claims.** The book's deployment stories are composite teaching examples, labeled as the book labels them, and its audit figures are illustrative readings, labeled illustrative. Routing, context discipline, and incremental indexing are presented as per-class contracts to be measured on your system — never as a promised saving, accuracy gain, or security outcome.

- **No compliance crosswalk or certification.** The Part 2 gate, the security audit board, and the appendix checklist are the book's own production audit artefacts. None of them maps controls to a framework, and none certifies anything.

- **No guarantee of security or correctness.** The book's own position: the security tests carry pass conditions of zero because disclosure is unforgiving — a test standard, not a promise — and a catalogue names, measures, and schedules failure classes; it does not eliminate them.

- **Not the architecture build pack.** The companion RAG Architecture Templates & Checklists pack (Product #2) covers how to build the pipeline. This pack assumes that pipeline exists and audits how its prompt, evaluation, security, cost, and organisational decisions fail in production. The two packs do not restate each other.

### Companion products

**RAG Anti-Patterns — Part 1: Ingest, Chunking, Embedding, Index & Retrieval Sins** (Product #14) is this pack's sibling, built from Chapters 1–6 of the same book: a companion for the pipeline half of the catalogue — never a requirement for this pack to work. The **RAG Architecture Templates & Checklists** pack (Product #2) is the build-side companion — the architecture this pack deliberately audits rather than restates. This pack stands alone.

*Note: the $99 Architecture Templates Bundle currently contains Products #1–#3 only. This pack launches standalone; no bundle inclusion is promised for this product.*

Built by the author of **RAG Anti-Patterns**, from the failure catalogue behind production retrieval systems.

### About the author

Naveen Kumar Josyula is the author of 35 AI books on Amazon KDP and the CEO and founder of AISoltrics Technologies Inc. He is the author of the source book, *RAG Anti-Patterns*, from which every template in this pack is distilled.

### Support promise

If anything in this file is wrong or unclear, email support@aisoltrics.com before you judge it — we fix verified issues within one business day and send you the corrected file free. (You can also reach the founder's office at aadhya@aisoltrics.com.) Updates to this product are free to past buyers via your library. Sold by AISoltrics Technologies Inc.