The Evidence Lab
Instructor guide · Everyday Mysteries, Better Questions
Created by Yoav Duman · edition 2026-10-10.1. Fictional teaching cases. No account, timer, or score. This guide explains the actual cases; it is intended for instructors.
What students should be able to do
Offer two plausible explanations of the same observation, choose evidence that distinguishes them, and limit a conclusion to what a comparison or sequence establishes. The software checks choices and response length; an instructor assesses the reasoning.
| Module | Distinct reasoning move | What to look for |
|---|---|---|
| Plants · 2 steps | Starting conditions and a fair experimental comparison. | Growth differs from final height; vary daylight while making other care comparable. |
| Café · 3 steps | Confounding through a different mix of visits. | Use within-period rates and group composition. The observational table does not establish a ban’s effect. |
| Navigation app · 3 steps | Temporal order and a response mistaken for a cause. | A later event cannot cause an earlier first wrong turn. Sequence alone establishes neither motive nor the app’s later usefulness. |
| Your mystery · 2 steps · Full | Design a study, state differing predictions, then critique it. | The rivals explain the same outcome but predict different observable patterns. Improve a specific weakness; do not invent findings. |
| Public power · 1 step · Full | Policy effects and political explanations. | Distinguish estimating a ban’s effect from explaining why governments adopt different rules. Similar-place comparisons still need assumptions about other changes. |
| Battery saver · 1 step | Near transfer without multiple-choice cues. | Connect a rival explanation to discriminating evidence; state what remains unknown. |
Choose an assignment
Core: warm-up, café, app, and battery saver: 9 required steps. For 50 minutes, start with 3 minutes of introduction, about 30 of play, 12 of debrief, and 5 for export. A fresh class link can end in ?route=core. Full: all 12 steps; also construct a study and consider public authority. Use ?route=full or select Full on Start. Plan 45–60 minutes of play with discussion afterward, or assign across two sessions. These are planning estimates, not measured student timings.
The selected route controls navigation, completion, and the report. Changing it preserves previously entered answers. An existing browser draft keeps its route when reopened; a conflicting class link displays a notice instead of replacing work. Ask students to verify their route on Start and in their report.
Worked reasoning example · a different case
A fictional library finds that visitors wearing headphones stay longer. One explanation is that headphones reduce distractions and enable longer work. A rival is that visitors planning long work sessions are more likely to bring headphones. A comparison that randomly offers similar visitors headphones, with an agreed measure of work time, can test the first mechanism; assignment, take-up, noise conditions, and measurement still matter. Recording plans before arrival helps investigate selection, but is not equivalent to random assignment.
A weak response: “Ask a thousand people whether headphones help.” More observations can repeat the same selection problem. A defensible response: name the process, identify the starting difference, and explain why the proposed comparison addresses it. A result can favor one account without proving every part of it.
Debrief · reconstruct the argument
- Explain one case to someone who has not seen the interface: observation, two accounts, useful evidence, conclusion, and remaining uncertainty.
- Why did the café’s more detailed comparison change the interpretation? What could the app timeline rule out, and what could it not establish?
- Give a political analogy—such as emergency assistance and unrest. Specify what transfers and what additional evidence that analogy needs.
Avoid making “correlation is not causation” the entire lesson. Students must explain the particular alternative and the evidence needed to investigate it.
Assess the report
Use four criteria, each 0–2: accurate use of the observation (0 missing/incorrect, 1 partly correct, 2 precise); plausible competing processes; a comparison with differing predictions; and a bounded conclusion with remaining uncertainty. For each remaining criterion, 0 means missing or contradictory, 1 means named but incompletely connected, and 2 means specific and connected to the case. Full adds the student’s own design and critique as evidence for these same criteria.
Do not grade word counts as understanding. Correct radio choices are practice checks, and retries are allowed. Reports preserve the first checked response when revised, identify opened hints, and keep a first battery-saver attempt before help. That challenge offers near-transfer evidence; it does not demonstrate delayed retention or broad critical-thinking improvement. A justified answer that remains unchanged is legitimate.
Pause, export, and submit
Progress saves in the current browser when storage is available. Use the displayed status. Back up before switching device, browser, website, or file location; a JSON backup restores progress, while PDF or text is a readable submission. Legacy version-1 backups remain supported. They are labeled when their help use was not tracked. Autosave pauses if another tab changes the draft; resolve the conflict under How this works after backing up. An unreadable stored draft is preserved until the user explicitly replaces it.
At the end, print/save a PDF or download the text report and submit it through Canvas as assigned. Exporting does not submit. Names and responses remain in the browser or downloaded files; they are not sent to a grading service.
Access and pilot
Use native keyboard controls, text evidence tables, expandable hints, and untimed writing. Offer reading support without treating it as a conceptual hint. Screen-reader, phone-browser, and observed student completion testing are still needed; automated checks cannot establish those experiences. Protect debrief time and shorten the assignment through Core rather than deleting distinct reasoning moves.