Behavioral Research: Statistical Methods

Valid forms of reliability:

• Test-retest
• Construct
• Inter-rater
• Parallel forms
• Ecological

A ratio scale supports:

• Ordering
• Add/subtract
• Multiply/divide
• Meaningful mean
• All of the above

True of the t-distribution:

• Heavier tails than Normal
• Used when σ unknown
• Shape depends on df
• Symmetric around 0
• Approaches Normal as df → ∞

Increase statistical power:

• ↑ sample size
• ↑ α from .01 to .05
• Within-subjects design
• Reduce measurement noise
• ↓ true effect size

Correct about p-hacking:

• Optional stopping inflates Type I
• Trying many analyses and reporting favorable one is p-hacking
• Pre-registration helps
• Reporting only the most interesting outcome is fine
• Selectively dropping participants until p < .05 is p-hacking

Assumptions of OLS regression (LINeM):

• Linearity
• Independence of errors
• Homoscedasticity
• Predictors must be normal
• Normality of residuals

Inflate family-wise Type I error:

• Many independent t-tests on same data
• Pre-specifying one hypothesis
• Testing 50 outcomes
• Many subgroup analyses
• Bonferroni correction

Correct Bayes-rule term mappings:

• P(H) is prior
• P(D|H) is posterior
• P(D|H) is likelihood
• P(H|D) is posterior
• P(D) is the marginal probability of the data

Non-parametric tests:

• Mann-Whitney U
• Independent t-test
• Kruskal-Wallis
• Friedman
• One-way ANOVA

Bayes Factor is preferred to p-value when:

• Evidence for null is of interest
• Optional stopping desired
• Want to incorporate prior knowledge
• Want to fail-to-reject H₀ definitively
• Effect size needs reporting

Correct about ANOVA:

• F is a ratio of variances
• ANOVA tests whether all means are equal
• ANOVA assumes equal variances
• Significant F identifies which groups differ
• Effect size is η²

Methods to deal with multicollinearity:

• Drop one of the correlated predictors
• Ridge regression
• Combine via PCA
• Increase α
• Centre/standardise variables

Apply to binomial:

• Discrete
• Two outcomes per trial
• Fixed n
• Constant p
• Trials independent

Threats to internal validity:

• History
• Maturation
• Selection
• Sampling from a single university
• Mortality / attrition

Limitations of frequentist NHST:

• Binary reject/fail-to-reject thinking
• Cannot quantify evidence for H₀
• Sensitive to optional stopping
• Provides explicit prior beliefs
• Often relies on threshold α = .05