1. Lighthouse

Level 6 · The correlation heatmap and the strongest pairs

The request

From Priya Shah, Head of Data

“For the daily KPI review: which of our daily numbers move together? Give me the full correlation heatmap for 2025 and, in a table, the three strongest pairs.”

Your goal

The starter builds daily, one row per day of 2025 with spend, sessions, conversions and orders. Compute its correlation matrix. Set result to a DataFrame with the three strongest pairs of different columns, by absolute correlation: columns pair (the two column names in alphabetical order joined by ' - ', e.g. 'orders - spend') and r (the correlation, rounded to 2), sorted by absolute r from strongest to weakest, index reset. Draw the matrix with sns.heatmap.

  • Use the orders DataFramenot done yet
  • Use the web_sessions DataFramenot done yet
  • Show exactly pair rnot done yet

Your code

Python

Loading the editor…

    ResultPreview of orders · press Run to see yours
    Preview of orders
    order_idcustomer_idorder_datestatuschannel
    10011252023-01-01 20:51:19completedweb
    10021562023-01-02 07:58:46completedweb
    1003222023-01-02 10:25:27completedmarketplace
    1004482023-01-02 11:21:51completedmarketplace

    Mote checks your result

    Press Run. Each row that matches the goal becomes a firefly in your lantern.

    DataFrames

    Loaded for you as pandas DataFrames, plus pd and np.

    orders12,000 rows · 7 columns
    • order_idinteger
    • customer_idinteger
    • order_datetimestamp
    • statusvarchar
    • channelvarchar
    • shipping_feedecimal
    • coupon_codevarchar
    First 5 rows of orders
    order_idcustomer_idorder_datestatuschannelshipping_feecoupon_code
    10011252023-01-01 20:51:19completedweb0.00NaN
    10021562023-01-02 07:58:46completedweb0.00NaN
    1003222023-01-02 10:25:27completedmarketplace4.99WELCOME10
    1004482023-01-02 11:21:51completedmarketplace4.99WELCOME10
    1005822023-01-02 11:43:36completedweb0.00WELCOME10
    web_sessions40,000 rows · 7 columns
    • session_idinteger
    • customer_idinteger
    • started_attimestamp
    • devicevarchar
    • sourcevarchar
    • pages_viewedinteger
    • convertedboolean
    First 5 rows of web_sessions
    session_idcustomer_idstarted_atdevicesourcepages_viewedconverted
    1NaN2023-01-01 23:02:24mobilepaid1false
    2182023-01-01 15:07:49desktoppaid1false
    3NaN2023-01-01 16:55:46desktopdirect7false
    4NaN2023-01-01 12:43:28mobileorganic3false
    5NaN2023-01-01 13:06:21mobilesocial2false
    marketing_spend3,276 rows · 3 columns
    • spend_datedate
    • channelvarchar
    • amountdecimal
    First 5 rows of marketing_spend
    spend_datechannelamount
    2023-01-01paid492.99
    2023-01-01email37.38
    2023-01-01social200.53
    2023-01-02paid709.13
    2023-01-02email64.13
    customers2,018 rows · 9 columns
    • customer_idinteger
    • first_namevarchar
    • last_namevarchar
    • emailvarchar
    • cityvarchar
    • statevarchar
    • signup_datedate
    • segmentvarchar
    • referral_sourcevarchar
    First 5 rows of customers
    customer_idfirst_namelast_nameemailcitystatesignup_datesegmentreferral_source
    1OwenHudsonowen.hudson@icloud.comSan FranciscoCA2022-06-02consumersearch
    2DouglasMendozadmendoza69@gmail.comAtlantaGA2022-06-02consumerNaN
    3HenryBrookshbrooks@lakesidecafe.comLos AngelesCA2022-06-04businessNaN
    4BrianGrahambgraham@bluecedarhomes.comSeattleWA2022-06-06businesspinterest
    5KyleJimenezkjimenez27@yahoo.comIndianapolisIN2022-06-07consumerNaN
    products120 rows · 8 columns
    • product_idinteger
    • namevarchar
    • categoryvarchar
    • subcategoryvarchar
    • unit_pricedecimal
    • unit_costdecimal
    • launched_ondate
    • is_activeboolean
    First 5 rows of products
    product_idnamecategorysubcategoryunit_priceunit_costlaunched_onis_active
    101Vale Velvet LoveseatFurnitureSofas1659.00NaN2023-09-11true
    102Alder Oak Coffee TableFurnitureTables919.00503.502024-05-16true
    103Orchard Rattan Dining ChairFurnitureChairs789.00429.872023-05-27true
    104Upland Oak BookcaseFurnitureStorage429.00214.702022-12-20true
    105Avery Leather Bed FrameFurnitureBeds449.00232.702021-08-31true
    order_items30,338 rows · 6 columns
    • order_item_idinteger
    • order_idinteger
    • product_idinteger
    • quantityinteger
    • unit_pricedecimal
    • discount_pctdecimal
    First 5 rows of order_items
    order_item_idorder_idproduct_idquantityunit_pricediscount_pct
    11001121931.990.00
    21001154311.990.00
    31002154411.990.00
    410021941149.000.00
    51002193219.990.00
    returns1,673 rows · 5 columns
    • return_idinteger
    • order_item_idinteger
    • returned_ondate
    • reasonvarchar
    • refund_amountdecimal
    First 5 rows of returns
    return_idorder_item_idreturned_onreasonrefund_amount
    1142023-01-10damaged83.69
    2422023-01-10wrong_item19.99
    31052023-01-14wrong_item10.99
    4242023-01-16changed_mind26.99
    51262023-01-16late20.99
    employees60 rows · 8 columns
    • employee_idinteger
    • first_namevarchar
    • last_namevarchar
    • departmentvarchar
    • titlevarchar
    • manager_idinteger
    • hire_datedate
    • salaryinteger
    First 5 rows of employees
    employee_idfirst_namelast_namedepartmenttitlemanager_idhire_datesalary
    1CatherineHaleExecutiveChief Executive OfficerNaN2016-03-01285000
    2RobertKimExecutiveChief Operating Officer12017-01-09235000
    3MeganCollinsSalesHead of Sales12018-04-16172000
    4ChloeGutierrezSalesAccount Executive32020-01-2172000
    5FrankBellSalesAccount Executive32023-05-2173000

    The lesson

    The whole matrix at once

    DataFrame.corr() computes r for every pair of numeric columns and returns a square table:

    matrix = daily.corr().round(2)
    #              spend  sessions  conversions  orders
    # spend         1.00      0.71         0.47    0.67
    # sessions      0.71      1.00         0.66    0.84
    # conversions   0.47      0.66         1.00    0.58
    # orders        0.67      0.84         0.58    1.00

    Two things are always true: the diagonal is 1 (every column correlates perfectly with itself), and the table is symmetric (spend with sessions equals sessions with spend). Leave identifier columns like order_id out: a correlation with an id number means nothing.

    As a heatmap

    Sixteen numbers are easier to scan as colors:

    ax = sns.heatmap(matrix, annot=True, fmt=".2f", cmap="coolwarm", vmin=-1, vmax=1,
                     linewidths=0.5, square=True)

    annot=True writes each value in its cell and fmt=".2f" rounds it. cmap="coolwarm" is a diverging palette: blue for negative, red for positive, pale near zero, which suits a number with a sign. vmin=-1, vmax=1 fixes the scale, so 0.5 always looks like 0.5. Without it, seaborn stretches the colors to your data and a weak 0.4 can look as hot as a strong 0.9.

    The strongest pairs, in code

    Reading a big matrix by eye misses things, so list the pairs. Each distinct pair is a cell above the diagonal, row i and column j with j > i:

    for i in range(len(cols)):
        for j in range(i + 1, len(cols)):
            ...matrix.iloc[i, j]...

    Rank them by absolute value, because -0.8 is as strong as +0.8, but keep the sign in the result: it carries the direction. In 2025’s daily numbers, sessions and orders go together most (0.84), followed by sessions and spend (0.71) and orders and spend (0.67). Conversions are only moderately tied to anything, since they’re a small number that jumps around from day to day.

    Common mistakes

    • Ranking by r instead of abs(r), which pushes strong negative pairs to the bottom.
    • Including the diagonal or both halves of the matrix, so every pair shows up twice.
    • A heatmap without vmin=-1, vmax=1, whose colors can’t be compared across charts.
    • Correlating ids, dates turned into numbers, or other columns that aren’t measurements.
    • Reporting only the strongest pairs. A pair close to zero is a finding too: it tells the team which numbers they can stop watching together.

    Pick what may run on this site. You can change this any time from "Cookie settings" at the bottom of every page.