1. Lighthouse

Level 6 · Boss: the first-look report

The request

From Priya Shah, Head of Data

“The CEO asked for a few headline facts for the board pack. How many customers have actually ordered, how orders split across channels, and what our price range looks like.”

Your goal

Step 1 of 3. Set result to the number of distinct customers who placed at least one order (count distinct customer_id in orders), as an int.

  • Use the orders DataFramenot done yet
  • Store a number in resultnot done yet

Step 1 · your code

Python

Loading the editor…

    ResultPreview of orders · press Run to see yours
    Preview of orders
    order_idcustomer_idorder_datestatuschannel
    10011252023-01-01 20:51:19completedweb
    10021562023-01-02 07:58:46completedweb
    1003222023-01-02 10:25:27completedmarketplace
    1004482023-01-02 11:21:51completedmarketplace

    Mote checks your result

    Press Run. Each row that matches the goal becomes a firefly in your lantern.

    Step 2 · your code

    Python

    Loading the editor…

      ResultPreview of orders · press Run to see yours
      Preview of orders
      order_idcustomer_idorder_datestatuschannel
      10011252023-01-01 20:51:19completedweb
      10021562023-01-02 07:58:46completedweb
      1003222023-01-02 10:25:27completedmarketplace
      1004482023-01-02 11:21:51completedmarketplace

      Mote checks your result

      Press Run. Each row that matches the goal becomes a firefly in your lantern.

      Step 3 · your code

      Python

      Loading the editor…

        ResultPreview of products · press Run to see yours
        Preview of products
        product_idnamecategorysubcategoryunit_price
        101Vale Velvet LoveseatFurnitureSofas1659.00
        102Alder Oak Coffee TableFurnitureTables919.00
        103Orchard Rattan Dining ChairFurnitureChairs789.00
        104Upland Oak BookcaseFurnitureStorage429.00

        Mote checks your result

        Press Run. Each row that matches the goal becomes a firefly in your lantern.

        DataFrames

        Loaded for you as pandas DataFrames, plus pd and np.

        orders12,000 rows · 7 columns
        • order_idinteger
        • customer_idinteger
        • order_datetimestamp
        • statusvarchar
        • channelvarchar
        • shipping_feedecimal
        • coupon_codevarchar
        First 5 rows of orders
        order_idcustomer_idorder_datestatuschannelshipping_feecoupon_code
        10011252023-01-01 20:51:19completedweb0.00NaN
        10021562023-01-02 07:58:46completedweb0.00NaN
        1003222023-01-02 10:25:27completedmarketplace4.99WELCOME10
        1004482023-01-02 11:21:51completedmarketplace4.99WELCOME10
        1005822023-01-02 11:43:36completedweb0.00WELCOME10
        products120 rows · 8 columns
        • product_idinteger
        • namevarchar
        • categoryvarchar
        • subcategoryvarchar
        • unit_pricedecimal
        • unit_costdecimal
        • launched_ondate
        • is_activeboolean
        First 5 rows of products
        product_idnamecategorysubcategoryunit_priceunit_costlaunched_onis_active
        101Vale Velvet LoveseatFurnitureSofas1659.00NaN2023-09-11true
        102Alder Oak Coffee TableFurnitureTables919.00503.502024-05-16true
        103Orchard Rattan Dining ChairFurnitureChairs789.00429.872023-05-27true
        104Upland Oak BookcaseFurnitureStorage429.00214.702022-12-20true
        105Avery Leather Bed FrameFurnitureBeds449.00232.702021-08-31true
        customers2,018 rows · 9 columns
        • customer_idinteger
        • first_namevarchar
        • last_namevarchar
        • emailvarchar
        • cityvarchar
        • statevarchar
        • signup_datedate
        • segmentvarchar
        • referral_sourcevarchar
        First 5 rows of customers
        customer_idfirst_namelast_nameemailcitystatesignup_datesegmentreferral_source
        1OwenHudsonowen.hudson@icloud.comSan FranciscoCA2022-06-02consumersearch
        2DouglasMendozadmendoza69@gmail.comAtlantaGA2022-06-02consumerNaN
        3HenryBrookshbrooks@lakesidecafe.comLos AngelesCA2022-06-04businessNaN
        4BrianGrahambgraham@bluecedarhomes.comSeattleWA2022-06-06businesspinterest
        5KyleJimenezkjimenez27@yahoo.comIndianapolisIN2022-06-07consumerNaN
        order_items30,338 rows · 6 columns
        • order_item_idinteger
        • order_idinteger
        • product_idinteger
        • quantityinteger
        • unit_pricedecimal
        • discount_pctdecimal
        First 5 rows of order_items
        order_item_idorder_idproduct_idquantityunit_pricediscount_pct
        11001121931.990.00
        21001154311.990.00
        31002154411.990.00
        410021941149.000.00
        51002193219.990.00
        returns1,673 rows · 5 columns
        • return_idinteger
        • order_item_idinteger
        • returned_ondate
        • reasonvarchar
        • refund_amountdecimal
        First 5 rows of returns
        return_idorder_item_idreturned_onreasonrefund_amount
        1142023-01-10damaged83.69
        2422023-01-10wrong_item19.99
        31052023-01-14wrong_item10.99
        4242023-01-16changed_mind26.99
        51262023-01-16late20.99
        employees60 rows · 8 columns
        • employee_idinteger
        • first_namevarchar
        • last_namevarchar
        • departmentvarchar
        • titlevarchar
        • manager_idinteger
        • hire_datedate
        • salaryinteger
        First 5 rows of employees
        employee_idfirst_namelast_namedepartmenttitlemanager_idhire_datesalary
        1CatherineHaleExecutiveChief Executive OfficerNaN2016-03-01285000
        2RobertKimExecutiveChief Operating Officer12017-01-09235000
        3MeganCollinsSalesHead of Sales12018-04-16172000
        4ChloeGutierrezSalesAccount Executive32020-01-2172000
        5FrankBellSalesAccount Executive32023-05-2173000
        web_sessions40,000 rows · 7 columns
        • session_idinteger
        • customer_idinteger
        • started_attimestamp
        • devicevarchar
        • sourcevarchar
        • pages_viewedinteger
        • convertedboolean
        First 5 rows of web_sessions
        session_idcustomer_idstarted_atdevicesourcepages_viewedconverted
        1NaN2023-01-01 23:02:24mobilepaid1false
        2182023-01-01 15:07:49desktoppaid1false
        3NaN2023-01-01 16:55:46desktopdirect7false
        4NaN2023-01-01 12:43:28mobileorganic3false
        5NaN2023-01-01 13:06:21mobilesocial2false
        marketing_spend3,276 rows · 3 columns
        • spend_datedate
        • channelvarchar
        • amountdecimal
        First 5 rows of marketing_spend
        spend_datechannelamount
        2023-01-01paid492.99
        2023-01-01email37.38
        2023-01-01social200.53
        2023-01-02paid709.13
        2023-01-02email64.13

        The lesson

        Headline facts need precise definitions

        “How many customers do we have?” sounds simple, but there are at least two answers in this dataset. len(customers) counts everyone who signed up. orders["customer_id"].nunique() counts everyone who actually placed an order. The difference, people who signed up and never bought, is a number marketing cares about a lot. When you report a headline figure, say which definition you used.

        Step 1: distinct versus total

        len(orders)                         # 12000 orders
        orders["customer_id"].nunique()     # distinct customers among them

        A customer with five orders appears five times in orders, so len overcounts customers. nunique() counts each ID once. This is one of the most frequent mistakes in junior reports, and one of the most frequent interview questions: “count customers, not orders”.

        Step 2: shares instead of counts

        Boards and managers want proportions, because they compare across time and across businesses of different sizes:

        orders["channel"].value_counts(normalize=True).round(3)

        normalize=True divides each count by the total. Round at the end: .round(3) keeps shares like 0.493, which is 49.3%. If you need percentages for a slide, multiply by 100 or format with f"{share:.1%}".

        Before reporting shares, check that the column has no missing values (orders["channel"].isna().sum()). value_counts drops missing values by default, so the shares would quietly add up to 1 over only the known rows.

        Step 3: a price summary

        products["unit_price"].describe().round(2)

        The result is a Series indexed by statistic name, so you can pull out single numbers:

        summary = products["unit_price"].describe()
        summary["50%"]     # the median price
        summary["max"]     # the most expensive product

        When you present this, lead with the median and the range, and mention that the mean is pulled up by expensive furniture. A summary that only says “average price $270” invites the wrong picture of a store where half the products cost under $154.

        The habit behind this level

        Whenever you get a new dataset, run this kind of profile before answering any question:

        1. Row counts and the grain of each table.
        2. Distinct counts of the key IDs.
        3. value_counts(dropna=False) on every category column.
        4. describe() on every numeric column.
        5. isna().sum() everywhere.

        It takes five minutes and catches most data problems before they reach a report: the duplicate customers, missing costs and odd quantities you’ll deal with in the cleaning chapter all show up in exactly these checks.

        Common mistakes

        • Using len(orders) or orders["customer_id"].count() for distinct customers. count() counts non-missing values, not distinct ones.
        • Forgetting normalize=True and returning counts.
        • Rounding the column before describe(), which changes the statistics. Round the summary instead.