Ugg hunt yes-or-no beast — straight spear no work

Ugg know old spear: Linear regression (OLS) — draw straight line through berry-cloud, guess how big Y get when X get big. Good when Y smooth, like how heavy mammoth. But sometimes Y have only two faces: beast caught or beast run. Admit or reject. Save or score. This called binary — Y is 0 or 1, nothing between. Straight spear SHATTER on this beast. Four ways it break.

Four ways straight spear break on yes-or-no

Ugg count on fingers:

1. Guess leak past edge. Line say $\hat{Y}={\beta }_0+{\beta }_{1}X$, but that can go below 0 or above 1. Chance-of-thing (probability) live ONLY between 0 and 1. Line say "134% chance admit." Ugg laugh — no such thing. 2. Left-over lump only two size. The miss (residual) can only be $0-\hat{p}$ or $1-\hat{p}$. Not smooth bell. Spear want smooth. No smooth here. 3. Wobble not same everywhere. For yes-no, wobble is $\text{Var}(Y)=p(1-p)$ — biggest at middle ($p=0.5$), tiny at edge. Spear want SAME wobble everywhere. Wobble change instead. Spear angry. 4. True path is S-snake, not straight. Push X near edge, chance move little. Push X in middle, chance move lots. Path bend like snake.

Fix not throw away regression. Fix: point spear at DIFFERENT thing that IS straight.

The two-jump trick

Ugg want number that live on whole line ($-\infty$ to $+\infty$) but made from chance $p$. Two jumps.

Jump one — chance to odds. Odds is win-times per lose-time: $\text{odds}=p/(1-p)$. Turn $(0,1)$ into $(0,\infty )$. Better, but still stuck above 0.

Jump two — odds to log. Take log: the logit (log-odds) $=\log (p/(1-p))$, run full $(-\infty ,\infty )$. Same scale as spear now! So Ugg write the logistic regression:

Ugg no press straight line on chance. Ugg press it on log-odds. Log-odds behave. Flip back to chance, get the logistic / sigmoid S-snake:

Flat low, steep at middle ($\eta =0$, $p=0.5$), flat high. Never leak past 0 or 1 — bounded by build. The S-snake Ugg wanted.

Read the spear-marks (β)

Two way to read β. Raw β — one step up in X ADD $\beta$ to log-odds (others held still). Clean, but no caveman think in log-odds.

Better: **$e^{\beta }$ is the odds ratio** — one step up in X MULTIPLY the odds by $e^{\beta }$. Tribe report this one.

• $e^{\beta }=2$ → odds double each step. $e^{\beta }=1$ → no change. $e^{\beta }=0.5$ → odds cut half. $e^{\beta }=1.05$ → odds up 5 in hundred.

Ugg warn: no do this! Odds ratio is NOT chance-ratio. OR $=2$ do NOT mean "twice as likely." Only near-true when $p$ small: for $p=0.1$, OR$=2$ push $p$ to about 0.18, not 0.20. Always say "odds ratio," NEVER "risk ratio."

Goalkeeper by the fire

Slide ask: keeper save more when tribe losing? Tribe behind: 24 kicks, 2 saved → $\text{odds(save)}=2/22\approx 0.091$. Not behind: 20 kicks, 6 saved → $\text{odds(save)}=6/14\approx 0.429$.

Behind multiply save-odds by 0.21 — chop 80 in hundred. Flip: $1/0.21\approx 4.7$ — keeper let ball in ~5 times more when tribe losing. Pressure break keeper. Data agree.

Find best β — most-likely, not least-miss

Straight spear pick β to make miss-squares small. Logistic no can — miss too weird. Instead use Maximum Likelihood Estimation (MLE): pick β so model give HIGHEST chance to the beast-tracks Ugg actually saw.

Multiply each track's model-chance together. Take log for steady numbers → log-likelihood. No neat answer-rock; fire-box grind it (Newton-Raphson, IRLS) till it settle. Ugg no do by hand. Ugg grunt at R: glm(admit ~ gre + gpa + factor(rank), family = binomial(link="logit")). Then exp(coef(model)) give odds ratios. Swap to family = poisson(link="log") → count-beast. Same spear, different beast.

One big family — the GLM

All these one tribe: Generalised Linear Model (GLM). Three parts: Random part = shape of Y (Normal, Bernoulli yes-no, Poisson count). Systematic part = the straight predictor $\eta =X\mathbf{\beta }$. Link g = bridge joining middle-of-Y to η: $g(E[Y])=\eta$.

Pick parts: smooth Y + identity link = old linear regression. Yes-no + logit = logistic. Count Y + log link = Poisson regression ($e^{\beta }$ = rate-ratio, assume Var=Mean; if Var much bigger, use negative binomial). Many-unordered Y = multinomial. Ordered Y = ordinal (proportional odds). One roof. Everything is GLM.

Rules Ugg obey, traps Ugg dodge

Assumptions:(1) Y binary.(2) tracks independent.(3) straight IN LOG-ODDS, not in chance — add bend-terms if snake.(4) no bad multicollinearity (predictors not twins — check VIF, same as OLS).(5) big tribe: ≥10 wins ($Y=1$) per predictor, count the RARER face. NO need: Normal Y, Normal residual, same-wobble — Bernoulli handle those.

Ugg warn: no do this!

• No read β as CHANCE change — β move log-odds, $e^{\beta }$ move odds.
• No use OLS $R^2$. Use McFadden pseudo R² $=1-\ln L_{\text{model}}/\ln L_{\text{null}}$, where 0.2 already GREAT (different scale, no compare to OLS).
• No trust only accuracy when one face rare — 95% right but useless if say "no" always. Use AUC, F1, recall.
• Compare nested models with likelihood ratio test: $D=-2(\ln L_{\text{red}}-\ln L_{\text{full}})\sim {\chi }_{\Delta k}^2$ (anova(m1,m2,test="Chisq")), NOT F-test.
• Perfect separation — one predictor split beast clean → β run to $\pm \infty$, huge wobble. That SICK fit, not great fit. Use Firth medicine.

Judge the hunt — classify beast

Turn chances to yes/no with a cut (default 0.5). Build confusion matrix (TP, FP, TN, FN). Then Accuracy $=(TP+TN)/N$, Precision $=TP/(TP+FP)$, Recall / sensitivity $=TP/(TP+FN)$. The ROC curve draw sensitivity against $(1-\text{specificity})$ as cut slide; AUC = area under = chance a random win out-ranks a random loss. 0.5 = coin, 1.0 = perfect.

Ugg remember

• Yes-no Y break straight spear four ways: leak edge, lumpy residual, changing wobble, S-snake. Model the log-odds instead — $\log (p/(1-p))=\eta$, flip to $p=1/(1+e^{-\eta })$.
• Report odds ratio $e^{\beta }$ (with 95% band), not raw β, and NEVER call it risk ratio — only close when $p$ tiny.
• Fit by MLE (glm(..., family=binomial)), not least-squares.
• GLM = random shape + straight η + link. Linear, logistic, Poisson, multinomial all one family — just pick shape and link.
• Logistic tools DIFFER: pseudo R² (0.2 = great), LRT not F-test, AUC/F1 when faces uneven, and watch for perfect separation.