Saral Shiksha Yojna
Courses/Behavioral Research: Statistical Methods

Behavioral Research: Statistical Methods

CG3.402
Vinoo AlluriMonsoon 2025-264 credits

The Case for Statistics — Biases, Base Rates, Bayes

NotesStoryCaveman
Unit 1 — Why Do Statistics? (Biases & Base Rates)

Brain lie to Ugg. Rock-math catch lie.

Ugg tell you fire-secret: your own head trick you. Head believe thing because it SOUND nice, not because it true. This why tribe make statistics — the rock-math for finding if thing that LOOKS true really IS true, when brain built to fool you. Exam ask "why do statistics?" — THIS the answer.

Meet Maya. She study people in Hyderabad. People hard! People complex (mood depend on sleep, weather, fight with mother, what they ate), variable (same person different Tuesday than Friday), and reactive (you watch them, they change). Maya mother say "turmeric milk cure sore throat, three day gone." Maya ask: true, or mother fool herself?

Brain-traps by the fire

Belief bias — Ugg judge argument by whether the ending SOUND right, not whether the logic right (Evans, Barston & Pollard, 1983 — remember this one name!). Say: "All Bengaluru engineer wear blue. This person wear blue. So this person a Bengaluru engineer." Sound fine, but BROKEN logic. ~70% of tribe accept it because it sound plausible; flip the ending to silly ("a Martian?") and they reject fast — belief, not logic, drove them.

Confirmation bias — when Ugg test idea, Ugg hunt only berries that AGREE, not berries that BREAK idea. Fire-test — Wason card task. Rule: "if card odd number one side, vowel other side." Cards: A, 2, 7, K. Most flip A. Wrong — flip A only can agree! To truly test, flip cards that could BREAK it: the 7 (consonant behind = rule dead) and the K (odd number behind = rule dead). Ugg warn: science is breaking your idea, not petting it. Answer = 7 + K.

Simpson's paradox — trend in each small tribe FLIP when you mash all tribes into one pile. UC Berkeley, 1973: whole-school look like it reject women. But department-by-department, women get in same or MORE! Trick: women applied to hard departments (low accept for EVERYONE), men to easy ones. Ugg warn: check small groups before trusting the big pile.

Base-rate fallacy — Ugg forget how RARE a thing is when a "positive test" come. Maya friend get positive mammogram. Chance she truly sick? Most people — even most doctor — shout "80–90%!" Wrong. Count stones: imagine 1000 women. Only ~8 truly have cancer (this the base rate — how common the thing is). Test catch 90%, so ~7 caught. Other 992 healthy, but test cry wolf on 7% → ~70 false alarm. Total positive = 7 + 70 = 77. Of those 77, only 7 truly sick. Chance = 7/77 ≈ 9%, NOT 90%.

Bayes rock — the fix

The proper rock-math is Bayes' rule:

Caveman words: chance-idea-true-AFTER-seeing-clue = (chance-of-clue-if-idea-true × chance-of-idea-BEFORE) ÷ chance-of-clue-any-way — after = fit × before ÷ all-ways. The bottom add up every way the clue can show. Mammogram: . Why so tiny? False-alarm pile () way bigger than true-catch pile (). Ugg trick: when thing rare, even good test mostly catch false alarm — low base rate → low PPV (positive predictive value = chance truly sick GIVEN positive). Ugg warn: never swap P(positive-if-sick) with P(sick-if-positive) — different rock!

Ugg warn: four big mix-ups.(1) p-value = P(data this wild | H₀ true), NOT chance H₀ true.(2) Confidence interval — the METHOD catches truth 95% of hunts; this one interval either hold it or not.(3) Statistical power belong to the TEST, not the chance the patient sick.(4) Correlation is not causation.

Maya build hunt-trap (experiment)

Independent variable (IV) = what Ugg CHANGE on purpose (turmeric vs plain milk); modern word predictor. Dependent variable (DV) = what Ugg MEASURE (throat change); modern word outcome. Control the change → can claim CAUSE. Only watch → only claim things move together.

Trap-shapes: between-subjects — different people each group. Within-subjects — SAME people try both; more power, each own control, BUT beware carry-over effect (first drink still working during second). Mixed design — some between, some within.

Confound — sneaky third thing tied to BOTH predictor and outcome, make fake footprint. "Violent games cause aggression?" Maybe those kids also get less parent-watching — parent-support is the confound. Best fix: random assignment (throw people to groups by chance, confound average out). Cheap fix: put confound in the model as covariate (ANCOVA, regression).

More traps that ruin hunt

Watch: history (event mid-study), maturation (people tire, drift), testing/practice, selection bias (groups differ BEFORE), attrition (non-random drop-out), non-response (only carers answer), regression to the mean (extreme score drift to middle — flight instructor THINK punishment work, really just drift), experimenter bias (Clever Hans horse read trainer face, Pfungst 1907), Hawthorne effect (change because watched), placebo, fraud.

Two exam-monsters: p-hacking (run 50 tests, show only the winner — no correction = lie) and HARKing (idea fail, mine data, write new finding as if predicted). Publication bias — journals print only exciting positive results; boring "nothing happen" rot in the file drawer → cave-library look like everything work → replication crisis (~36–47% of psychology studies replicate, OSC 2015). Antidote: pre-register BEFORE the hunt.

Double-blind — neither person NOR experimenter know who got what until data counted. Kill experimenter bias AND reactivity in one throw. Add placebo control to kill placebo. Last fire-word: statistics is NOT certainty — every claim leans on assumptions. State them, be calibrated (talk small when evidence small). Confident-wrong shaman worse than none.

Ugg remember

  • Why statistics? Brain fool itself about chance (belief, confirmation, Simpson, base-rate). Statistics is the fool-catcher.
  • Bayes rock: . Rare thing → low PPV, mostly false alarm.
  • Count, then divide. Draw the 1000-people table before touching fractions.
  • Wason = 7 + K. Break the rule, don't pet it: falsify, not confirm. And subgroup-check before trusting any big average.
  • Double-blind beats experimenter bias + reactivity; random assignment beats confounds; pre-register beats p-hacking.
End of storyUnit 1 — Why Do Statistics? (Biases & Base Rates) · The Case for Statistics — Biases, Base Rates, Bayes