Concept dictionary
Evaluation Gap
An evaluation gap is the difference between what an evaluation would need to test to support a real-world claim and what the evaluation actually measures.
- Review state
- Reviewed in POC
- Freshness
- Current
- Version
- 1
Also called
- Evaluation Gap
- AI Evaluation Gap
Remember it this wayThe score answers less than the claim.
In plain language
A score usually covers a slice of behavior under chosen data, metrics, users, and conditions. The evaluation gap is what remains untested when someone stretches that score into a broader claim about usefulness, safety, fairness, or readiness for a particular setting.
Why it matters
A narrow evaluation can create false confidence even when its numbers are accurate. Missing languages, users, failure modes, tool interactions, time pressure, or real consequences can matter more than the measured benchmark once the system is deployed.
Example
A support assistant scores well on factual accuracy for clean English questions. It is then called production-ready even though no evaluation covered multilingual users, prompt injection, tool failures, response latency, or harmful refusals in the actual support workflow.
Common misconception
An evaluation gap is not simply a low score or a lack of many tests. A large benchmark suite can still leave a serious gap if it measures the wrong conditions or values for the claim being made. The gap is between coverage and claim, not between a score and perfection.
Related terms
Sources
- Hutchinson et al., “Critical Evaluation Gaps in Machine Learning Practice”documents the gap between the broad concerns needed for responsible system use and the narrow predictive behaviors commonly evaluated.
- NIST AI RMF Playbook, “Manage”calls for evaluation under deployment-like conditions and continuing post-deployment monitoring as context and risk change.
Learn more
- Write the deployment claim first, then use [[verification|Verification]] to ask what evidence would actually support it.
- Record tested and untested conditions through [[capability-provenance|Capability Provenance]] and a [[model-card|Model Card]].