A cross-cutting evaluation reckoning is underway: aggregate accuracy and agreement-based metrics systematically mask failure modes, forcing a shift toward structural, full-sequence, and baseline-controlled evals.
Evidence
- Consistency-accuracy dissociation shows 60% of wrong answers look highly consistent, defeating label-free agreement signals
- ‘What Accuracy Hides’ and silent-failures-in-data-science demonstrate token accuracy and end-task benchmarks conceal complete failures and methodological flaws like data leakage
- Jev-as-a-Judge reveals identical-accuracy systems can differ fundamentally in false-positive vs false-negative patterns
- Uncertainty/perplexity is only a reliable hallucination signal with sufficient retrieval coverage, and zero-ablation tests credit even irrelevant documents
Implications
- Builders must add failure-mode-specific diagnostics (error-type breakdowns, full-output validation, proper no-memory baselines, CUJ-anchored multi-turn evals) rather than trusting single aggregate scores
- Cost-optimization and architecture decisions driven by headline accuracy risk shipping systems with hidden, divergent error profiles
Concepts
LLM Evals RAG Evaluation RAG Agents
Confidence
high