Chapter 4 · 6 levels
Correlation and regression
Measure how two columns move together, fit a straight line with slope and R², and catch the traps (confounders, extrapolation, Simpson's paradox) on Harbor & Pine data.
Start“Do these two things move together?” is one of the questions analysts get most often. Do pricier products sell fewer units? Does marketing spend bring in revenue? The tools are short to write (a correlation is one method call, a regression line one more) and easy to misread.
In this chapter you measure correlation on Harbor & Pine’s products and see why a straight-line measure can understate a curved relationship. Then you meet the confounder that sits behind the store’s most quoted chart, fit a regression line to monthly spend and revenue and read its slope and R² for what they are, and learn why a line should never be followed past the data it was fitted on. Before the boss you scan a whole correlation matrix at once, read it as a heatmap, and pick out the pairs that are worth a closer look.
The boss is Simpson’s paradox on the store’s own customers: one marketing source looks worse overall and better inside every segment. You’ll explain why, and compute the fair comparison.
Levels
- Do pricier products sell less?
from Jessica Turner, Merchandising Manager
correlation, pearson, rank correlation, scatter plot
- Does marketing spend drive revenue?
from Jordan Reyes, Head of Marketing
correlation vs causation, confounder, seasonality
- Fit a line to spend and revenue
from Brian Mitchell, Finance Analyst
linear regression, slope, intercept, r squared
- Can the line forecast a bigger budget?
from Jordan Reyes, Head of Marketing
extrapolation, r squared, prediction, linear regression
- Which product numbers move together?
from Priya Shah, Head of Data
correlation matrix, heatmap, correlation, association
- Boss: the channel that looks worse
from Jordan Reyes, Head of Marketing
simpsons paradox, confounder, segment mix, standardization, pivot table