LLM App RAG & Evals Prompt Pack

AI will build you a RAG demo in an afternoon — and then tell you recall is 92% and it costs $0.002 a query. Nobody measured either.

Ask ChatGPT, Claude, Cursor, Claude Code or Codex to "add RAG to our docs" and you get a loader, a splitter, a vector store and a prompt that says "use the context". It answers the three questions you try. Then users find mixed product versions, part numbers that never match, prompt tweaks that silently broke refusals, and citations to passages that were never retrieved.

LLM App RAG & Evals Prompt Pack gives you 32 ready-to-paste prompts that make the model work from your corpus, real questions and measurements, state its assumptions, refuse to invent accuracy, recall, hallucination rates, prices, token counts, latencies or model facts (anything model- or vendor-specific is marked VERIFY), treat retrieved text as data, not instructions, and hand you reviewable output: schemas, rubrics, test-case tables, eval harness sketches and go/no-go tables.

What you get

- PDF guide (46 pages, A4): why AI-built LLM features fail quietly, the LLM feature context block, all 32 prompts with when-to-use notes, combinations, a fictional worked example, prompt anatomy and follow-up lines

- templates/: 32 plain-Markdown prompts + 3 checklists + README

The 32 prompts

- Framing (01–04): LLM feature context block · does this need an LLM? · testable feature spec · explain an existing pipeline

- RAG design (05–11): corpus audit · chunking tradeoffs · metadata & permission filters · embedding model tradeoffs · vector index/store options · hybrid retrieval, rerank, query rewrite · retrieval failure analysis

- Prompts, schemas & tools (12–17): system prompt design · prompt-change review · context assembly & citations · structured output schema · tool-calling schema · output validation & fallbacks

- Evals (18–25): eval plan · golden set · synthetic cases (labelled) · LLM-as-judge rubric · judge calibration · retrieval evals · regression suite · results analysis

- Safety & grounding (26–29): grounding/hallucination check · prompt-injection threat model · injection & guardrail tests · refusal & fallback behaviour

- Ops & release (30–32): cost/latency framing (no invented numbers) · observability & tracing · release go/no-go

Also included

- RAG design review checklist

- Eval suite readiness checklist

- LLM feature release go/no-go checklist

What's NOT included

- Agent rule files for Cursor/Claude Code/Codex → AI Coding Agent Workflow Pack

- Service boundaries and ADRs → AI System Design & Architecture Prompt Pack

- General vulnerability hunting → AI Security Review Prompt Pack

- Unit tests for ordinary code → AI Test Generation Prompt Pack

- CI YAML, SQL, API contracts → sibling packs

- Fabricated benchmarks, model rankings, prices, testimonials or case studies — none appear in this pack

Accuracy

Techniques named are real and documented (structure-aware and parent-child chunking, BM25, hybrid search with reciprocal rank fusion, cross-encoder reranking, HNSW/IVF indexes, recall@k/MRR/nDCG, LLM-as-judge, Cohen's kappa, JSON Schema, OWASP Top 10 for LLM Applications, OpenTelemetry). None is a guarantee. Worked examples are labelled fictional.

Price: $19

Author: Pluto · X @Mars_Peakul

Edition: 1.0 · October 2026