1. Lighthouse
  2. Lighthouse

Level 2 · Cost per conversion in pandas

The request

From Jordan Reyes, Head of Marketing

“These numbers are going in the budget deck. Our marketing analyst works in notebooks, so please build the three tables in pandas, matching the SQL version of this case, so she can rerun them every month.”

Your goal

Step 1 of 3. Set result to a DataFrame with columns source, sessions, conversions and conversion_rate_pct (conversions * 100 / sessions, rounded to 2), sorted by conversion_rate_pct descending, with a default index.

  • Use the web_sessions DataFramenot done yet
  • Show exactly source sessions conversions conversion_rate_pctnot done yet
  • 5 rows, sorted as askednot done yet

Step 1 · your code

Python

Loading the editor…

    ResultPreview of web_sessions · press Run to see yours
    Preview of web_sessions
    session_idcustomer_idstarted_atdevicesource
    1NaN2023-01-01 23:02:24mobilepaid
    2182023-01-01 15:07:49desktoppaid
    3NaN2023-01-01 16:55:46desktopdirect
    4NaN2023-01-01 12:43:28mobileorganic

    Mote checks your result

    Press Run. Each row that matches the goal becomes a firefly in your lantern.

    Step 2 · your code

    Python

    Loading the editor…

      ResultPreview of web_sessions · press Run to see yours
      Preview of web_sessions
      session_idcustomer_idstarted_atdevicesource
      1NaN2023-01-01 23:02:24mobilepaid
      2182023-01-01 15:07:49desktoppaid
      3NaN2023-01-01 16:55:46desktopdirect
      4NaN2023-01-01 12:43:28mobileorganic

      Mote checks your result

      Press Run. Each row that matches the goal becomes a firefly in your lantern.

      Step 3 · your code

      Python

      Loading the editor…

        ResultPreview of web_sessions · press Run to see yours
        Preview of web_sessions
        session_idcustomer_idstarted_atdevicesource
        1NaN2023-01-01 23:02:24mobilepaid
        2182023-01-01 15:07:49desktoppaid
        3NaN2023-01-01 16:55:46desktopdirect
        4NaN2023-01-01 12:43:28mobileorganic

        Mote checks your result

        Press Run. Each row that matches the goal becomes a firefly in your lantern.

        DataFrames

        Loaded for you as pandas DataFrames, plus pd and np.

        web_sessions40,000 rows · 7 columns
        • session_idinteger
        • customer_idinteger
        • started_attimestamp
        • devicevarchar
        • sourcevarchar
        • pages_viewedinteger
        • convertedboolean
        First 5 rows of web_sessions
        session_idcustomer_idstarted_atdevicesourcepages_viewedconverted
        1NaN2023-01-01 23:02:24mobilepaid1false
        2182023-01-01 15:07:49desktoppaid1false
        3NaN2023-01-01 16:55:46desktopdirect7false
        4NaN2023-01-01 12:43:28mobileorganic3false
        5NaN2023-01-01 13:06:21mobilesocial2false
        marketing_spend3,276 rows · 3 columns
        • spend_datedate
        • channelvarchar
        • amountdecimal
        First 5 rows of marketing_spend
        spend_datechannelamount
        2023-01-01paid492.99
        2023-01-01email37.38
        2023-01-01social200.53
        2023-01-02paid709.13
        2023-01-02email64.13
        customers2,018 rows · 9 columns
        • customer_idinteger
        • first_namevarchar
        • last_namevarchar
        • emailvarchar
        • cityvarchar
        • statevarchar
        • signup_datedate
        • segmentvarchar
        • referral_sourcevarchar
        First 5 rows of customers
        customer_idfirst_namelast_nameemailcitystatesignup_datesegmentreferral_source
        1OwenHudsonowen.hudson@icloud.comSan FranciscoCA2022-06-02consumersearch
        2DouglasMendozadmendoza69@gmail.comAtlantaGA2022-06-02consumerNaN
        3HenryBrookshbrooks@lakesidecafe.comLos AngelesCA2022-06-04businessNaN
        4BrianGrahambgraham@bluecedarhomes.comSeattleWA2022-06-06businesspinterest
        5KyleJimenezkjimenez27@yahoo.comIndianapolisIN2022-06-07consumerNaN
        products120 rows · 8 columns
        • product_idinteger
        • namevarchar
        • categoryvarchar
        • subcategoryvarchar
        • unit_pricedecimal
        • unit_costdecimal
        • launched_ondate
        • is_activeboolean
        First 5 rows of products
        product_idnamecategorysubcategoryunit_priceunit_costlaunched_onis_active
        101Vale Velvet LoveseatFurnitureSofas1659.00NaN2023-09-11true
        102Alder Oak Coffee TableFurnitureTables919.00503.502024-05-16true
        103Orchard Rattan Dining ChairFurnitureChairs789.00429.872023-05-27true
        104Upland Oak BookcaseFurnitureStorage429.00214.702022-12-20true
        105Avery Leather Bed FrameFurnitureBeds449.00232.702021-08-31true
        orders12,000 rows · 7 columns
        • order_idinteger
        • customer_idinteger
        • order_datetimestamp
        • statusvarchar
        • channelvarchar
        • shipping_feedecimal
        • coupon_codevarchar
        First 5 rows of orders
        order_idcustomer_idorder_datestatuschannelshipping_feecoupon_code
        10011252023-01-01 20:51:19completedweb0.00NaN
        10021562023-01-02 07:58:46completedweb0.00NaN
        1003222023-01-02 10:25:27completedmarketplace4.99WELCOME10
        1004482023-01-02 11:21:51completedmarketplace4.99WELCOME10
        1005822023-01-02 11:43:36completedweb0.00WELCOME10
        order_items30,338 rows · 6 columns
        • order_item_idinteger
        • order_idinteger
        • product_idinteger
        • quantityinteger
        • unit_pricedecimal
        • discount_pctdecimal
        First 5 rows of order_items
        order_item_idorder_idproduct_idquantityunit_pricediscount_pct
        11001121931.990.00
        21001154311.990.00
        31002154411.990.00
        410021941149.000.00
        51002193219.990.00
        returns1,673 rows · 5 columns
        • return_idinteger
        • order_item_idinteger
        • returned_ondate
        • reasonvarchar
        • refund_amountdecimal
        First 5 rows of returns
        return_idorder_item_idreturned_onreasonrefund_amount
        1142023-01-10damaged83.69
        2422023-01-10wrong_item19.99
        31052023-01-14wrong_item10.99
        4242023-01-16changed_mind26.99
        51262023-01-16late20.99
        employees60 rows · 8 columns
        • employee_idinteger
        • first_namevarchar
        • last_namevarchar
        • departmentvarchar
        • titlevarchar
        • manager_idinteger
        • hire_datedate
        • salaryinteger
        First 5 rows of employees
        employee_idfirst_namelast_namedepartmenttitlemanager_idhire_datesalary
        1CatherineHaleExecutiveChief Executive OfficerNaN2016-03-01285000
        2RobertKimExecutiveChief Operating Officer12017-01-09235000
        3MeganCollinsSalesHead of Sales12018-04-16172000
        4ChloeGutierrezSalesAccount Executive32020-01-2172000
        5FrankBellSalesAccount Executive32023-05-2173000

        The lesson

        Same question, same definitions

        This is Jordan’s budget analysis in pandas, with the same definitions as the SQL level of this case: 2025 only on both tables, a conversion is a session with converted == True, and spend is matched to sessions by channel name. Organic and direct have sessions but no spend.

        Step 1: counting True values

        converted is a boolean column. In pandas True behaves as 1 and False as 0, so ("converted", "sum") counts conversions and ("converted", "mean") would give the rate directly. The task asks for a percentage rounded to 2, so compute it from the two counts:

        by_source = ws.groupby("source", as_index=False).agg(
            sessions=("session_id", "count"), conversions=("converted", "sum")
        )
        by_source["conversion_rate_pct"] = (by_source["conversions"] * 100 / by_source["sessions"]).round(2)

        "count" counts non-missing values of the named column. session_id is never missing, so it counts every session. Counting customer_id would be a quiet bug: anonymous sessions have NaN there and would not be counted.

        Step 2: pre-aggregate, then merge

        In SQL this takes two CTEs. In pandas each one is simply a DataFrame:

        conv = ws.groupby("source", as_index=False).agg(conversions=("converted", "sum"))
        spend = ms.groupby("channel", as_index=False).agg(spend=("amount", "sum")).rename(columns={"channel": "source"})
        table = conv.merge(spend, on="source", how="left", validate="one_to_one")

        marketing_spend calls its key channel while web_sessions calls it source. .rename(columns={"channel": "source"}) renames a column (old name on the left, new name on the right) so both frames share the key and on="source" works. left_on="source", right_on="channel" does the same job without renaming.

        validate="one_to_one" is the pandas way of asserting what you reasoned about in SQL: after the groupby, each side has one row per key, so the merge cannot fan out. If someone later merges the raw tables by mistake, pandas raises a MergeError instead of handing back inflated spend.

        how="left" from the sessions side keeps organic and direct, with NaN spend (pandas’ marker for a missing number). Any arithmetic with NaN gives NaN, so their cost per conversion is missing, which is correct: it is not zero, it is not applicable.

        Division by zero

        pandas silently returns inf (infinity) when you divide a number by zero. The fix is to turn zeros in the denominator into missing values first. .replace(0, np.nan) swaps every 0 in a Series for np.nan, numpy’s missing number, and anything divided by NaN is NaN:

        pd.Series([5000, 5000]) / pd.Series([25, 0])                    # 200.0, inf
        pd.Series([5000, 5000]) / pd.Series([25, 0]).replace(0, np.nan)  # 200.0, NaN

        No 2025 channel or month has zero conversions in this data, but a monthly report that runs every month will meet one eventually, and inf in a chart is hard to explain.

        Step 3: add month to both keys

        Build a month label on each table with .dt.strftime("%Y-%m"), which turns a datetime into text like "2025-03" (%Y is the year, %m the two-digit month). Group each table by month, and merge on month. The grain is now one row per month on both sides.

        SQL and pandas side by side

        Step SQL pandas
        conversions COUNT(*) FILTER (WHERE converted) ("converted", "sum")
        CTE WITH conv AS (...) conv = ws.groupby(...)
        key names differ ON s.channel = c.source rename or left_on/right_on
        assert no fan-out reason about it validate="one_to_one"
        NULLIF(x, 0) built in x.replace(0, np.nan)

        Pitfalls

        • Merging raw sessions with raw spend. Even on ["source", "date"], each day’s spend repeats once per session that day.
        • Rounding spend before dividing. Compute cost per conversion from the raw sum, then round.
        • Missing spend days. Twelve outage rows are absent from marketing_spend; the totals are a slight undercount, the same in SQL and pandas.
        • Dates as strings. The filters compare a datetime64 column with "2025-01-01"; pandas parses the string, so no conversion is needed.