When the Average Lies: Simpson's Paradox in AI Evaluation

Help advanced learners recognize when a correct aggregate metric answers the wrong decision question. The packet uses synthetic AI-evaluation data to practice reading a claim, inspecting denominators and case mix, comparing meaningful strata, standardizing to a declared target population, and recording uncertainty before a recommendation.

Purpose

Help advanced learners recognize when a correct aggregate metric answers the wrong decision question. The packet uses synthetic AI-evaluation data to practice reading a claim, inspecting denominators and case mix, comparing meaningful strata, standardizing to a declared target population, and recording uncertainty before a recommendation.

The central discipline is:

An aggregate result can be calculated correctly and still answer the wrong decision question.

Imported Source Packet

Zai’s source packet was copied unchanged on 2026-07-14 from inbox/RADAR_Simpsons_Paradox_Learning_Lab_Packet. It is retained under source-packet:

  • two PDF guides, one for facilitators and one for learners;
  • learner and instructor workbooks;
  • synthetic CSV datasets for two lessons and a capstone; and
  • the original packet README, including its expected checks and reuse guidance.

The instructor workbook must remain separate from learner-facing material if this is ever piloted.

Proposed Learning Sequence

  1. The Winner Changes When We Slice the Data — a beginner-accessible reversal exercise using routine and complex cases.
  2. Evaluation Under Unequal Case Mix — an advanced analysis using synthetic record-level data, stratified rates, a declared production mix, and uncertainty discussion.
  3. The Dashboard Says Ship It — a capstone in which a deployment dashboard favors the wrong model because its evaluation mix underrepresents high-severity cases.

The sequence is an imported sketch, not a required track path, runnable class, or validated assessment.

RADAR Evaluation Check

  • Read the claim: State what the metric is being used to claim.
  • Audit the denominator and mix: Check who, what, and how much each group contributes to the total.
  • Disaggregate: Calculate performance within meaningful strata.
  • Adjust to a common population: Standardize only when the target mix is relevant and defensible.
  • Recommend and record uncertainty: Make a decision, document limitations, and specify the next evaluation.

Boundaries Before Any Learner Use

  • Confirm that the statistical explanation, calculations, and answer keys are correct and appropriate for the intended audience.
  • Preserve the packet’s causal caution: disaggregation and standardization can support a decision-relevant comparison without proving a causal effect.
  • Review the PDFs, worksheets, spreadsheet accessibility, timing, and facilitator cues with people other than the author.
  • Keep all scenario data clearly labeled as synthetic; do not treat it as evidence about a real model or deployment.
  • Decide whether the introductory reversal, advanced evaluation, and capstone should remain one sequence or become separate modules after a pilot.
  • Human ownership remains.
  • Do not automate confusion.
  • Trust, but check.
  • Record uncertainty rather than guessing.
  • Map the decision before improving the system.

Status Notes

The packet arrived more developed than an ideation note, so its source assets are preserved intact rather than converted into finished Learning Lab curriculum. The source assets were copied byte-for-byte; their presentation and calculations have not yet received independent review or learner testing.

Review status

  • Package lifecycle: Sketch
  • Learner availability: Deferred
  • Review needs: Imported source packet; requires statistical-methods, accessibility, learning-design, and pilot review

Review this module

Source: modules/when-the-average-lies-simpsons-paradox-in-ai-evaluation/module.md