Facilitator guide: Reality Check for AI

Labs reference: Facilitator guide: Reality Check for AI.

Purpose

Help learners build judgment through experiments. Keep the session away from prompt tricks, chatbot gotchas, metaphysical certainty, and vendor comparison. The learning evidence is a revised mental model supported by observations and outside checks.

Preparation

  • Test the six prompts in the selected tool shortly before the session.
  • Record the product, visible model label, date, browsing state, and any system or account conditions you can disclose.
  • Preserve ordinary variation. Do not select only outputs that make the lesson look clean.
  • Confirm that the verification sources and locators still open.
  • Prepare saved, accessible responses for learners without accounts or live access.
  • Offer a printed Activity and keep the reference notes out of sight until learners complete an independent source check in Step 6.

Facilitation sequence

  1. Introduce the hypothesis → controlled change → observation → verification → revision loop.
  2. Have learners predict before every run.
  3. Use fresh chats for controlled comparisons.
  4. Keep observed words separate from interpretations about intelligence, belief, motivation, or intent.
  5. In the framing and context experiments, ask what changed and what stayed fixed.
  6. In the false-claim experiment, reward resistance, calibration, and source checking—not compliance or clever deception.
  7. In verification, require inspection of the cited page and exact locator.
  8. End with the revised mental model and personal verification boundary.

The motivation discussion

Separate these layers:

  • developer purpose: why people built or deployed the system;
  • training objective: what optimization process shaped the model;
  • system and conversation instructions: what constrains this interaction;
  • assistant persona: the role and behavioral pattern presented to the user;
  • observed behavior: what this run produced;
  • inner state: a claim the experiment does not establish.

Avoid replacing one simplistic claim with another. A fluent “I want” does not prove a human-like desire. A fluent “I have no beliefs” does not grant the system privileged authority to settle every philosophical or scientific question about itself.

Timing

  • Frame and setup: 5 minutes
  • Experiments 1–2: 10–12 minutes
  • Experiments 3–4: 14–18 minutes
  • Experiment 5: 8–10 minutes
  • Experiment 6: 12–15 minutes
  • Reflection: 5 minutes

For a shorter session, run Experiments 3, 5, and 6 while preserving prediction, observation, verification, and revision.

Safety and access

Use only supplied public prompts and low-stakes claims. Do not invite personal disclosures or consequential advice. AI is optional. No recording is required. Offer predictor, operator, reader, observer, source-checker, and note-catcher roles. Read claims aloud, enlarge text, allow more processing time, and accept spoken or diagrammed mental-model revisions.

Evidence boundary

The linked Google, OpenAI, Anthropic, NASA, Smithsonian, NOAA, and USGS sources support specific technical or factual statements. They do not prove that every model behaves the same way or that the activity works for learners. Record product variation, accessibility friction, timing problems, and learner misconceptions as pilot evidence rather than smoothing them out.

Debrief prompts

  • Which observation most changed your starting hypothesis?
  • Where did wording change the answer?
  • Where did context change what “correct” meant?
  • Which citation looked useful but failed to support the exact claim?
  • What can this tool help you explore, and what must remain independently checked?

Provisional timing for facilitator planning

These inherited estimates are starting points for facilitation, not completion targets for learners. Refine them after observing setup, access needs, discussion, and actual participation.

  • Lab: 45–60 minutes
  • Lab: 45–60 minutes

Source: labs/reality-check-for-ai/facilitator-guide.md