Ugg ask: mammoth real, or just wind in grass?

Maya find berry-milk make throat-sick end in 4.2 day. But maybe milk do NOTHING and Ugg just lucky? To know for real, tribe play the big question game: Null Hypothesis Significance Testing (NHST) — the fire-side way to tell if a thing is REAL or just random luck of the hunt. Every later test is built from this one machine.

First the hunt-plan: big-idea not same as sharp-guess

Theory — big broad story about the world. "Warm-root calm the throat-fire." Whole forest.

Hypothesis — one sharp guess Ugg can TEST. "Berry-milk make sore throat shorter than plain water." One tree Ugg throw spear at. Ugg warn: exam LOVE this. Theory big and general. Hypothesis small and sharp.

The black-swan rule (Popper law)

Falsifiability — a real guess must be able to be PROVEN WRONG. Some footprint could exist that kill it. Ugg say "all swan white." See 100 white swan — that no PROVE it! One black swan, guess is dead meat. So Ugg never say "hunt PROVE my guess." Ugg only say "hunt not kill my guess yet." You cannot prove a forever-claim by counting; you only fail to kill it.

Two rocks: boring rock and shiny rock

Null hypothesis (H₀) — the boring rock. "Nothing happen. Milk do nothing."

Alternative hypothesis (H₁) — the shiny rock Ugg believe. "Milk make throat better."

Game: try to SMASH boring rock. If data would be super weird when boring rock true, smash it and grab shiny rock. If data look normal, cannot smash. Ugg warn BIG: never say "Ugg ACCEPT boring rock." Only "Ugg FAIL TO SMASH it." Rock may still be wrong — Ugg just no have enough spear yet.

How surprised is Ugg? The p-value

p-value — chance of seeing data THIS weird or weirder, IF boring rock is true.

p = 0.03 mean: "if milk truly do nothing, only 3-in-100 chance Ugg see result this big by dumb luck." Small p = big surprise = smash the boring rock. Ugg warn, this trap kill many hunter: p is NOT "chance boring rock is true." p is NOT "97% chance milk real." p speak only about DATA, assuming rock true. Never flip it around.

The line in the sand: alpha

α (alpha) — the surprise-line, and the Type-I risk Ugg agree to take. Tribe pick α = 0.05 by custom (physics use way stricter). Rule: $p<\alpha$ → smash H₀. $p\ge \alpha$ → cannot smash.

Two ways Ugg mess up

Type I error (α) — false alarm. Ugg scream "MAMMOTH!" but no mammoth. Smash a TRUE boring rock. Ugg pick this risk himself when pick α.

Type II error (β) — missed mammoth. Real mammoth come, Ugg no see, tribe get stomped. Fail to smash a FALSE boring rock.

The see-saw: make α smaller (stricter), β gets BIGGER — Ugg miss more real mammoth. To shrink BOTH at once: more hunters (bigger n), less noise, or a smarter same-people trap.

Power — the sharp eye

Power = 1 − β — chance Ugg CATCH the mammoth when mammoth truly there. Tribe want power ≥ 0.80. Power grows with FOUR things:(1) which test Ugg pick,(2) the α line,(3) effect size — how BIG the mammoth,(4) n — how many hunters. Bigger n → more power. Bigger effect → more power. Stricter α → less power. Power analysis done BEFORE the hunt, to know how many hunter Ugg need.

Big mammoth or tiny mouse? Effect size

Ugg warn: with HUGE tribe (n = 10,000) even a tiny 1-point change gives tiny p — "significant"! But 1 point is a MOUSE, not mammoth. Statistical significance (small p) is NOT the same as practical significance (change big enough to matter).

Cohen's d — how big the mammoth, in standard-step units:

Ugg read: gap between two group means, divided by the spread. Custom marks: $d=0.2$ small, $0.5$ medium, $0.8$ large. Ugg RULE: always tell effect size next to p! "p < 0.001, d = 0.12" mean real but tiny mouse.

Different spears for different hunts: t-tests

t-test — spear for comparing means. Pick by hunt:

One-sample t — one group vs a known number: $t=\frac{\bar{x}-{\mu }_0}{s/\sqrt{n}}$, df = n − 1.

Independent t — two DIFFERENT tribes: $t=\frac{{\bar{x}}_1-{\bar{x}}_2}{\sqrt{s_p^2(1/n_1+1/n_2)}}$, df = n₁ + n₂ − 2.

Paired t — SAME people, before-after: $t=\frac{\bar{d}}{s_d/\sqrt{n}}$, df = n − 1. Paired spear sharper than independent for same n — it throws away the between-people noise.

Welch's t — like independent but two tribes have different spread: $t=\frac{{\bar{x}}_1-{\bar{x}}_2}{\sqrt{s_1^2/n_1+s_2^2/n_2}}$. Modern default when unsure.

One tail or two: two-tailed (default) cares about change either way. One-tailed cares one direction, puts full 0.05 in one tail — easier to smash but BLIND to the other side. Ugg warn: pick one-tail only BEFORE the hunt. Switch to one-tail after seeing data to rescue p = 0.06 is cheating — p-hacking!

Rope and stake: CI same as test

Confidence interval (CI) and the test are two faces of one rock. If the 95% CI leaves OUT the "no-effect" number, the two-tail test smashes H₀ at α = 0.05. If CI wraps around it, cannot smash. Maya CI [3.82, 4.58] leaves out 5 days → smash.

Ugg warn: no do these bad-hunter tricks

• Say p = chance rock true. NO — p speaks of data, given rock.
• "Accept" boring rock. NO — only "fail to smash."
• Switch to one-tail to save p = 0.06. Cheating.
• Give p with no effect size or CI. Broken report.
• Ignore the test rules (normal-ish data, independent, equal spread). Bad hunt.

Ugg remember

• Boring rock (H₀) = "no effect." Never ACCEPT it — only FAIL TO SMASH. Smash when $p<\alpha$ (custom α = 0.05).
• p-value = P(data this weird | H₀) — chance of the data IF rock true. NOT the chance the rock is true!
• Type I = false alarm (α). Type II = missed mammoth (β). Power = 1 − β, want ≥ 0.80. Stricter α trades for more miss; only bigger n / less noise shrink both.
• Effect size (Cohen's d: 0.2 / 0.5 / 0.8) always sits beside p — big tribe make a tiny mouse look "significant."
• Pick spear by hunt: one-sample, independent, paired (sharper, same people), Welch (unequal spread). CI leaves out the null ↔ test smashes.