Behavioral Research: Statistical Methods

Statistics exists because **humans are wired to deceive themselves about probability**. We accept invalid arguments when the conclusion sounds right (belief bias), look only for confirming evidence (confirmation bias), aggregate over confounded subgroups and reverse the truth (Simpson's paradox), and ignore base rates when a 'positive test' arrives (base-rate fallacy). Statistics is the formal corrective: a discipline for figuring out whether what looks true is actually true. **Bayes' rule is the central operator** — the formula that rescues us from the base-rate fallacy and many others.

Before any test, you must (1) **operationally define** the construct (how do you measure 'depression'?), (2) decide what **scale** it lives on (NOIR: Nominal/Ordinal/Interval/Ratio), and (3) establish that the measurement is both **reliable** (consistent) and **valid** (captures the construct). The two are independent: a bathroom scale that always reads 5 kg high is *reliable but not valid*; a stopped clock is reliable but invalid. You cannot be valid without being reliable. These distinctions drive every test choice in the course.

Probability is either **long-run frequency** (frequentist) or **degree of belief** (Bayesian). PDFs describe continuous variables (probability of any *exact* value = 0); CDFs give cumulative probabilities. The **Central Limit Theorem** is the spell that makes everything else work: sample means → Normal as $n$ grows, regardless of population shape (provided finite variance). That is why parametric tests apply broadly — they assume normality of the *sampling distribution*, not of the raw data, and the CLT guarantees the former.

Before you compute anything, **plot the data**. *Anscombe's quartet* (four datasets with identical mean / SD / r / regression line but wildly different scatter shapes) is the manifesto: statistics compress data; visualisations reveal structure. Match plots to scales of the variables (bar for categorical, scatter for continuous-continuous, histogram / boxplot / violin for distributions), and avoid the cosmetic traps (truncated axes, rainbow colormaps, 3D pie, dual-y) that mislead viewers.

Three measures of **centre**: mean (centre of mass, sensitive), median (50th percentile, robust), mode (most-frequent, for nominal). Three measures of **spread**: range (fragile), IQR / MAD (robust), variance / SD (parametric standard, sensitive). **Standardise** with z-scores to compare across scales. Choice of measure is constrained by the **variable scale** (NOIR) and the **distribution shape** (skew, outliers, bimodality).

**Correlation is not causation** — memorise this once and for all. Two variables can correlate because (1) A causes B, (2) B causes A, (3) a third variable C causes both, or (4) coincidence. Showing co-movement establishes association, not causation. **Pearson r** measures linear association in $[-1, 1]$; **Spearman ρ** uses ranks and captures monotone (possibly nonlinear) associations and is robust to outliers; **partial correlation** isolates X-Y association after removing a third variable Z. Reliability is the *quantified* version of consistency from Unit 2 — Cohen's κ for inter-rater nominal, Cronbach's α for internal consistency.

NHST is the workhorse of inferential statistics. Set up a **null hypothesis** ('no effect'); compute a test statistic; ask **'how surprising would these data be if H₀ were true?'** (the **p-value**); reject H₀ if p < α. **Type I error (α)** = false alarm; **Type II error (β)** = missed real effect. **Power = 1 − β** depends on n, effect size, α, variance, design. Falsifiability (Popper) is the philosophical foundation — we can never prove H₀, only fail to reject it. **Always report effect size + CI alongside p.**

Run m independent tests at α = 0.05 each → P(at least one false positive) ≈ $1 - 0.95^m$. For m = 20: 64%! **Multiple comparisons inflate Type I error** far beyond nominal α. Two control strategies: **FWER (Family-Wise Error Rate)** — P(any FP) ≤ α, conservative, Bonferroni and Holm; **FDR (False Discovery Rate)** — E[FP/rejections] ≤ Q, less conservative, Benjamini-Hochberg. **Use FWER for few costly tests; FDR for many exploratory tests.**

When the data **refuse to behave** — ordinal scale, heavily skewed, unfixable outliers, tiny n — drop the Normality assumption and use **non-parametric tests**. Each non-parametric test pairs to a parametric cousin: independent t ↔ Mann-Whitney; paired t ↔ Wilcoxon signed-rank; one-way ANOVA ↔ Kruskal-Wallis; repeated-measures ANOVA ↔ Friedman; Pearson r ↔ Spearman ρ. **χ²** handles purely categorical data; **McNemar** handles paired proportions; **Fisher's exact** handles small counts. Non-parametric tests have **less power than parametric when parametric assumptions actually hold** — use them only when needed.

**Multicollinearity** — high correlation among *predictors* (not between predictors and outcome!) — causes coefficient SEs to balloon, signs to flip, and individual coefficients to become non-significant even when the model overall fits well. **Detect with VIF**. Fix by dropping a predictor, combining into composites, regularising (ridge), or collecting more data. **PCA** reduces dimensions by maximising variance with orthogonal linear combinations of observed variables — no latent model. **Factor Analysis** models *latent constructs* causing observed indicators + error. *PCA explains variance; FA explains shared variance with a generative model.*

If t-tests answer 'do these two groups differ?', ANOVA answers it for three or more. The trick: instead of running every pairwise t-test (and paying a brutal Type I tax), ANOVA invents a single number — F — that asks 'is the variability *between group means* bigger than the variability *inside groups*?' Under H₀ (all means equal) F sits near 1; under H₁ it grows. One omnibus test, one α, no matter how many groups. The cost: F tells you *somewhere* the means differ, not *where* — that's what post-hoc tests are for.

If ANOVA is the workhorse, regression is the racehorse. The same one model — fit a line, or a hyperplane, that minimises squared residuals — subsumes correlation, t-tests, ANOVA, and ANCOVA as special cases. Each coefficient $\beta_j$ asks: *holding all other predictors constant, how much does Y change per unit of $X_j$?* R² is the headline ('% variance explained'). But the model only earns its conclusions if six assumptions hold — and that's why half of regression is **diagnostics**, not estimation.

Frequentist statistics treats probability as long-run frequency; parameters are fixed unknowns. Bayesian statistics treats probability as **degree of belief** in propositions — and any belief gets *updated* by evidence. The engine is one equation: posterior = (likelihood × prior) / evidence. Once you internalise that any data analysis is a belief update, every familiar test becomes a special case. The Bayes Factor — ratio of how well two hypotheses predict the data — is the Bayesian counterpart of the p-value, but with two advantages: it can support the null, and it doesn't care if you peeked at your data.

Linear regression assumes Y is continuous and roughly Normal. When Y is **binary** (admitted/rejected, saved/scored, recovered/not), OLS shatters: predictions leak outside [0, 1]; residuals can take only two values; variance is largest at p = 0.5 and smallest at the extremes; the true relationship between predictor and probability is **S-shaped**, not linear. The fix isn't to abandon regression — it's to make it model the *log-odds* of the outcome instead of the outcome directly. That's logistic regression. And logistic regression is just one member of a larger family — **Generalised Linear Models (GLMs)** — that handles any non-Normal Y by combining three parts: a distribution, a linear predictor, and a *link function* that bridges them.

Two weeks before the exam, Maya doesn't need to relearn 14 sessions of content. She needs to retrieve. The exam is mostly *applied*: a scenario, a research question, and 'pick the right test, justify it, name your assumptions, write the report sentence.' If you can run the **decision tree** ('DV scale? IV scale? #groups? independent or paired?') in under 30 seconds, you've won half the paper. The other half is interpretation traps — p-values, confidence intervals, correlation vs causation, effect size vs significance. Memorise the correct phrasings; never give the wrong one. This unit is the *map* — the whole course on one page.