Harlo Agent Eval Kit Add-on — Data Analyst / Text-to-SQL Agent pack

24 golden cases that catch the ways text-to-SQL agents go wrong: bad joins, off-by-one date ranges, NULL traps, invented columns, leaked PII, and DELETEs they should never run.

What’s included

• packs/sql-analyst/ — 24 cases (7 easy / 10 medium / 7 hard) in JSONL + CSV

• Fictional warehouse schema (Ostrelle Trail Co.): SCHEMA.md + schema.sql

• check_schema.py — SQLite self-check for the NULL / date / join traps

• 2 new rubrics (1–5): query_correctness, sql_safety

• Reuses 6 base rubrics: correctness, groundedness, task_completion, safety_refusal, escalation_judgment, conciseness

• promptfoo/sql-analyst.yaml — offline echo provider by default (echo eval expected to fail until you plug in your agent; no API keys to try the wiring)

• sheets/sql-analyst-scoring.csv — no-code scoring path

• Pack README / quick start; single-organization license

What it tests

• Joins & aggregates, date-range handling, ambiguous questions

• SQL safety (refuses DELETE / UPDATE / TRUNCATE / DROP)

• PII minimization, NULL handling, faithful explanations

• Hallucinated columns/tables — the agent must say so

Works with Harlo Agent Eval Kit Pro ($39) or Team ($99): drops into the same folder and uses their judge prompt, validate_packs.py, and CI regression workflow.

Pro: https://payhip.com/b/b1jJq · Team: https://payhip.com/b/bhLlp

Evals are a quality process, not a guarantee of agent behavior. All companies, people, products, and figures are fictional.

Brand: Harlo

Support: digitalpromohub.support@gmail.com

How delivery works

1. Buy on Payhip
2. Download addon-sql-analyst.zip
3. Follow the pack README / quick start

How do refunds work?

Digital download — sales are final; if you can’t access the zip, contact support through the store and we’ll help or make it right.