Concept dictionary
Evidence Selection
Evidence selection is the process of deciding which observations, records, sources, examples, or test results will be used to evaluate a claim or support a decision.
- Review state
- Reviewed in POC
- Freshness
- Current
- Version
- 1
Also called
- evidence-selection
Remember it this wayThe evidence you choose sets the boundary of what you can conclude.
In plain language
A conclusion depends partly on what evidence is allowed into the analysis. Good selection seeks relevant, reliable, and sufficiently representative evidence, including material that could challenge the preferred answer.
Why it matters
Careful reasoning cannot rescue a distorted evidence set. Selection choices can hide failures, overrepresent convenient cases, or make a system appear safer and more capable than it is in real use.
Example
A team evaluates an AI support tool using only short, well-structured questions written by its developers. It excludes long, ambiguous requests from actual customers. The resulting accuracy score describes the selected test set, not the full service environment.
Common misconception
More evidence is not automatically better evidence. Repeated copies of one claim do not create independent support, and a large convenient sample can still omit the people, conditions, or counterexamples that matter.
Related terms
Sources
- NIST, “AI RMF Core: Measure”representative evaluation conditions, documented test sets, meaningful measurement, and consultation with affected actors.
- NIST, “Artificial Intelligence Risk Management Framework 1.0”context-dependent measurement, population differences, baselines, and risks in real-world settings.