1. Lighthouse

Level 2 · Check and build the web funnel in pandas

The request

From Jordan Reyes, Head of Marketing

“We agreed the plan: a funnel by device, sessions to engaged to converted, then the same rates by year. Before you build it, tell me what the session data can be trusted for. I've heard half our visitors are anonymous.”

Your goal

Step 1 of 3. Check the data. Set result to a one-row DataFrame with columns sessions (all rows), distinct_ids (distinct session_id), anonymous (customer_id is null), anonymous_converted (anonymous and converted), empty_sessions (pages_viewed < 1), converted_under_3_pages (converted with pages_viewed < 3) and converted (all converted sessions).

  • Use the web_sessions DataFramenot done yet
  • Show exactly sessions distinct_ids anonymous anonymous_converted empty_sessions converted_under_3_pages convertednot done yet
  • 1 row, sorted as askednot done yet

Step 1 · your code

Python

Loading the editor…

    ResultPreview of web_sessions · press Run to see yours
    Preview of web_sessions
    session_idcustomer_idstarted_atdevicesource
    1NaN2023-01-01 23:02:24mobilepaid
    2182023-01-01 15:07:49desktoppaid
    3NaN2023-01-01 16:55:46desktopdirect
    4NaN2023-01-01 12:43:28mobileorganic

    Mote checks your result

    Press Run. Each row that matches the goal becomes a firefly in your lantern.

    Step 2 · your code

    Python

    Loading the editor…

      ResultPreview of web_sessions · press Run to see yours
      Preview of web_sessions
      session_idcustomer_idstarted_atdevicesource
      1NaN2023-01-01 23:02:24mobilepaid
      2182023-01-01 15:07:49desktoppaid
      3NaN2023-01-01 16:55:46desktopdirect
      4NaN2023-01-01 12:43:28mobileorganic

      Mote checks your result

      Press Run. Each row that matches the goal becomes a firefly in your lantern.

      Step 3 · your code

      Python

      Loading the editor…

        ResultPreview of web_sessions · press Run to see yours
        Preview of web_sessions
        session_idcustomer_idstarted_atdevicesource
        1NaN2023-01-01 23:02:24mobilepaid
        2182023-01-01 15:07:49desktoppaid
        3NaN2023-01-01 16:55:46desktopdirect
        4NaN2023-01-01 12:43:28mobileorganic

        Mote checks your result

        Press Run. Each row that matches the goal becomes a firefly in your lantern.

        DataFrames

        Loaded for you as pandas DataFrames, plus pd and np.

        web_sessions40,000 rows · 7 columns
        • session_idinteger
        • customer_idinteger
        • started_attimestamp
        • devicevarchar
        • sourcevarchar
        • pages_viewedinteger
        • convertedboolean
        First 5 rows of web_sessions
        session_idcustomer_idstarted_atdevicesourcepages_viewedconverted
        1NaN2023-01-01 23:02:24mobilepaid1false
        2182023-01-01 15:07:49desktoppaid1false
        3NaN2023-01-01 16:55:46desktopdirect7false
        4NaN2023-01-01 12:43:28mobileorganic3false
        5NaN2023-01-01 13:06:21mobilesocial2false
        customers2,018 rows · 9 columns
        • customer_idinteger
        • first_namevarchar
        • last_namevarchar
        • emailvarchar
        • cityvarchar
        • statevarchar
        • signup_datedate
        • segmentvarchar
        • referral_sourcevarchar
        First 5 rows of customers
        customer_idfirst_namelast_nameemailcitystatesignup_datesegmentreferral_source
        1OwenHudsonowen.hudson@icloud.comSan FranciscoCA2022-06-02consumersearch
        2DouglasMendozadmendoza69@gmail.comAtlantaGA2022-06-02consumerNaN
        3HenryBrookshbrooks@lakesidecafe.comLos AngelesCA2022-06-04businessNaN
        4BrianGrahambgraham@bluecedarhomes.comSeattleWA2022-06-06businesspinterest
        5KyleJimenezkjimenez27@yahoo.comIndianapolisIN2022-06-07consumerNaN
        products120 rows · 8 columns
        • product_idinteger
        • namevarchar
        • categoryvarchar
        • subcategoryvarchar
        • unit_pricedecimal
        • unit_costdecimal
        • launched_ondate
        • is_activeboolean
        First 5 rows of products
        product_idnamecategorysubcategoryunit_priceunit_costlaunched_onis_active
        101Vale Velvet LoveseatFurnitureSofas1659.00NaN2023-09-11true
        102Alder Oak Coffee TableFurnitureTables919.00503.502024-05-16true
        103Orchard Rattan Dining ChairFurnitureChairs789.00429.872023-05-27true
        104Upland Oak BookcaseFurnitureStorage429.00214.702022-12-20true
        105Avery Leather Bed FrameFurnitureBeds449.00232.702021-08-31true
        orders12,000 rows · 7 columns
        • order_idinteger
        • customer_idinteger
        • order_datetimestamp
        • statusvarchar
        • channelvarchar
        • shipping_feedecimal
        • coupon_codevarchar
        First 5 rows of orders
        order_idcustomer_idorder_datestatuschannelshipping_feecoupon_code
        10011252023-01-01 20:51:19completedweb0.00NaN
        10021562023-01-02 07:58:46completedweb0.00NaN
        1003222023-01-02 10:25:27completedmarketplace4.99WELCOME10
        1004482023-01-02 11:21:51completedmarketplace4.99WELCOME10
        1005822023-01-02 11:43:36completedweb0.00WELCOME10
        order_items30,338 rows · 6 columns
        • order_item_idinteger
        • order_idinteger
        • product_idinteger
        • quantityinteger
        • unit_pricedecimal
        • discount_pctdecimal
        First 5 rows of order_items
        order_item_idorder_idproduct_idquantityunit_pricediscount_pct
        11001121931.990.00
        21001154311.990.00
        31002154411.990.00
        410021941149.000.00
        51002193219.990.00
        returns1,673 rows · 5 columns
        • return_idinteger
        • order_item_idinteger
        • returned_ondate
        • reasonvarchar
        • refund_amountdecimal
        First 5 rows of returns
        return_idorder_item_idreturned_onreasonrefund_amount
        1142023-01-10damaged83.69
        2422023-01-10wrong_item19.99
        31052023-01-14wrong_item10.99
        4242023-01-16changed_mind26.99
        51262023-01-16late20.99
        employees60 rows · 8 columns
        • employee_idinteger
        • first_namevarchar
        • last_namevarchar
        • departmentvarchar
        • titlevarchar
        • manager_idinteger
        • hire_datedate
        • salaryinteger
        First 5 rows of employees
        employee_idfirst_namelast_namedepartmenttitlemanager_idhire_datesalary
        1CatherineHaleExecutiveChief Executive OfficerNaN2016-03-01285000
        2RobertKimExecutiveChief Operating Officer12017-01-09235000
        3MeganCollinsSalesHead of Sales12018-04-16172000
        4ChloeGutierrezSalesAccount Executive32020-01-2172000
        5FrankBellSalesAccount Executive32023-05-2173000
        marketing_spend3,276 rows · 3 columns
        • spend_datedate
        • channelvarchar
        • amountdecimal
        First 5 rows of marketing_spend
        spend_datechannelamount
        2023-01-01paid492.99
        2023-01-01email37.38
        2023-01-01social200.53
        2023-01-02paid709.13
        2023-01-02email64.13

        The lesson

        What you are checking, and why

        web_sessions is a tracking table: every visit to the site, its device and source, how many pages it saw and whether it ended in a purchase. Tracking tables are full of holes, so before you build a funnel on one, find out what it can and cannot show. Step 1 asks four questions in one pass:

        • Is any session counted twice? Compare len(s) with s["session_id"].nunique().
        • Are there empty sessions? A visit with zero pages is a tracking glitch.
        • Who can convert? Count anonymous sessions, and how many of them converted.
        • Does the definition hold? Count conversions with fewer than three pages.

        A condition on a column gives a boolean Series, and summing it counts the True values:

        anon = s["customer_id"].isna()
        (anon & s["converted"]).sum()   # 0

        The zero is the most important number in the case. More than half the sessions can never show a purchase, because purchases are only recorded against an account. The rates you build next are rates of recorded purchases, and the funnel needs a stage for signing in. Wrapping each count in int() turns NumPy integers into plain numbers for the one-row DataFrame.

        A funnel is a chain of boolean columns

        A funnel follows the same sessions through stages that narrow down: all sessions, engaged (pages_viewed >= 3), engaged and signed in, converted. In pandas each stage is a boolean column, and named aggregation sums them per device in one call:

        result = s.groupby("device", as_index=False).agg(
            sessions=("session_id", "count"),
            engaged=("engaged", "sum"),
            known_engaged=("known_engaged", "sum"),
            converted=("converted", "sum"),
        )

        The rates divide a stage by the one before it, so you can see where people drop out: engaged_pct is the browsing step, close_pct is the checkout step.

        Reading the funnel

        device engaged_pct close_pct conversion_pct
        desktop 44.1 21.4 3.87
        mobile 44.1 12.9 2.41
        tablet 44.8 14.6 2.81

        The browsing step is the same on every device. The checkout step is where mobile falls behind. That changes the brief for the mobile team: the product pages are doing their job; the basket, the sign-in and the payment form are where to look.

        Why the year view matters

        Step 3 repeats the conversion rate by year, one column per device. A neat trick: multiply a boolean by 100 and its mean is a percentage. unstack() then turns the device level of the index into columns. Mobile’s share of traffic grew every year, and mobile converts less, so the overall rate is a weighted average whose weights keep moving towards the low rate. Every device improved from 2023 to 2025, while the overall rate grew much less. If you kept 2023’s device mix and applied 2025’s rates, the overall rate would be about 3.24% instead of 3.08%. Always split a rate by its segments before you call it a trend.

        Pitfalls

        • == None and == np.nan never match; use .isna() and .notna().
        • & binds tighter than <, so wrap each comparison in brackets.
        • Dividing by sessions for the close rate mixes stages: divide by the stage just before.

        Pick what may run on this site. You can change this any time from "Cookie settings" at the bottom of every page.