1. Lighthouse

Level 3 · A category summary table

The request

From Thomas Whitaker, Head of Merchandising

“For the range review I want one small table: for each category, how many products we carry, the average price and the most expensive price.”

Your goal

Set result to a DataFrame with one row per category and columns category, n_products (number of products), avg_price (mean unit_price, rounded to 2) and max_price (max unit_price), sorted by category A to Z, with category as a regular column (not the index).

  • Use the products DataFramenot done yet
  • Show exactly category n_products avg_price max_pricenot done yet
  • 6 rows, sorted as askednot done yet

Your code

Python

Loading the editor…

    ResultPreview of products · press Run to see yours
    Preview of products
    product_idnamecategorysubcategoryunit_price
    101Vale Velvet LoveseatFurnitureSofas1659.00
    102Alder Oak Coffee TableFurnitureTables919.00
    103Orchard Rattan Dining ChairFurnitureChairs789.00
    104Upland Oak BookcaseFurnitureStorage429.00

    Mote checks your result

    Press Run. Each row that matches the goal becomes a firefly in your lantern.

    DataFrames

    Loaded for you as pandas DataFrames, plus pd and np.

    products120 rows · 8 columns
    • product_idinteger
    • namevarchar
    • categoryvarchar
    • subcategoryvarchar
    • unit_pricedecimal
    • unit_costdecimal
    • launched_ondate
    • is_activeboolean
    First 5 rows of products
    product_idnamecategorysubcategoryunit_priceunit_costlaunched_onis_active
    101Vale Velvet LoveseatFurnitureSofas1659.00NaN2023-09-11true
    102Alder Oak Coffee TableFurnitureTables919.00503.502024-05-16true
    103Orchard Rattan Dining ChairFurnitureChairs789.00429.872023-05-27true
    104Upland Oak BookcaseFurnitureStorage429.00214.702022-12-20true
    105Avery Leather Bed FrameFurnitureBeds449.00232.702021-08-31true
    customers2,018 rows · 9 columns
    • customer_idinteger
    • first_namevarchar
    • last_namevarchar
    • emailvarchar
    • cityvarchar
    • statevarchar
    • signup_datedate
    • segmentvarchar
    • referral_sourcevarchar
    First 5 rows of customers
    customer_idfirst_namelast_nameemailcitystatesignup_datesegmentreferral_source
    1OwenHudsonowen.hudson@icloud.comSan FranciscoCA2022-06-02consumersearch
    2DouglasMendozadmendoza69@gmail.comAtlantaGA2022-06-02consumerNaN
    3HenryBrookshbrooks@lakesidecafe.comLos AngelesCA2022-06-04businessNaN
    4BrianGrahambgraham@bluecedarhomes.comSeattleWA2022-06-06businesspinterest
    5KyleJimenezkjimenez27@yahoo.comIndianapolisIN2022-06-07consumerNaN
    orders12,000 rows · 7 columns
    • order_idinteger
    • customer_idinteger
    • order_datetimestamp
    • statusvarchar
    • channelvarchar
    • shipping_feedecimal
    • coupon_codevarchar
    First 5 rows of orders
    order_idcustomer_idorder_datestatuschannelshipping_feecoupon_code
    10011252023-01-01 20:51:19completedweb0.00NaN
    10021562023-01-02 07:58:46completedweb0.00NaN
    1003222023-01-02 10:25:27completedmarketplace4.99WELCOME10
    1004482023-01-02 11:21:51completedmarketplace4.99WELCOME10
    1005822023-01-02 11:43:36completedweb0.00WELCOME10
    order_items30,338 rows · 6 columns
    • order_item_idinteger
    • order_idinteger
    • product_idinteger
    • quantityinteger
    • unit_pricedecimal
    • discount_pctdecimal
    First 5 rows of order_items
    order_item_idorder_idproduct_idquantityunit_pricediscount_pct
    11001121931.990.00
    21001154311.990.00
    31002154411.990.00
    410021941149.000.00
    51002193219.990.00
    returns1,673 rows · 5 columns
    • return_idinteger
    • order_item_idinteger
    • returned_ondate
    • reasonvarchar
    • refund_amountdecimal
    First 5 rows of returns
    return_idorder_item_idreturned_onreasonrefund_amount
    1142023-01-10damaged83.69
    2422023-01-10wrong_item19.99
    31052023-01-14wrong_item10.99
    4242023-01-16changed_mind26.99
    51262023-01-16late20.99
    employees60 rows · 8 columns
    • employee_idinteger
    • first_namevarchar
    • last_namevarchar
    • departmentvarchar
    • titlevarchar
    • manager_idinteger
    • hire_datedate
    • salaryinteger
    First 5 rows of employees
    employee_idfirst_namelast_namedepartmenttitlemanager_idhire_datesalary
    1CatherineHaleExecutiveChief Executive OfficerNaN2016-03-01285000
    2RobertKimExecutiveChief Operating Officer12017-01-09235000
    3MeganCollinsSalesHead of Sales12018-04-16172000
    4ChloeGutierrezSalesAccount Executive32020-01-2172000
    5FrankBellSalesAccount Executive32023-05-2173000
    web_sessions40,000 rows · 7 columns
    • session_idinteger
    • customer_idinteger
    • started_attimestamp
    • devicevarchar
    • sourcevarchar
    • pages_viewedinteger
    • convertedboolean
    First 5 rows of web_sessions
    session_idcustomer_idstarted_atdevicesourcepages_viewedconverted
    1NaN2023-01-01 23:02:24mobilepaid1false
    2182023-01-01 15:07:49desktoppaid1false
    3NaN2023-01-01 16:55:46desktopdirect7false
    4NaN2023-01-01 12:43:28mobileorganic3false
    5NaN2023-01-01 13:06:21mobilesocial2false
    marketing_spend3,276 rows · 3 columns
    • spend_datedate
    • channelvarchar
    • amountdecimal
    First 5 rows of marketing_spend
    spend_datechannelamount
    2023-01-01paid492.99
    2023-01-01email37.38
    2023-01-01social200.53
    2023-01-02paid709.13
    2023-01-02email64.13

    The lesson

    Several numbers per group

    A summary table usually needs more than one statistic. agg computes several at once. The clearest form is named aggregation: each keyword is the name of an output column, and its value is a (column, function) pair.

    products.groupby("category").agg(
        n_products=("product_id", "count"),
        avg_price=("unit_price", "mean"),
        max_price=("unit_price", "max"),
    )

    You decide the column names up front, so there’s no renaming afterwards and no confusing multi-level headers. This is the style to use in shared code.

    The function can be a string ("sum", "mean", "median", "min", "max", "count", "size", "nunique", "first") or any function that takes a Series and returns one value:

    products.groupby("category").agg(
        price_range=("unit_price", lambda s: s.max() - s.min()),
    )

    Keeping the key as a column

    By default the group keys become the index. With as_index=False, they stay a regular column, which is what you want for a table you’ll sort, merge or export:

    products.groupby("category", as_index=False).agg(...)

    Calling .reset_index() after a normal groupby gives the same shape.

    Rounding selected columns

    DataFrame.round accepts a dictionary, so you can round only what needs it:

    summary.round({"avg_price": 2})

    Rounding the whole table with .round(2) is fine too, since counts are whole numbers anyway. Just avoid rounding the raw price column before averaging, which would change the result.

    Other aggregation styles you’ll see

    Older code and tutorials use other forms. They work, but produce less predictable column names:

    products.groupby("category")["unit_price"].agg(["mean", "max"])   # columns: mean, max
    products.groupby("category").agg({"unit_price": ["mean", "max"]}) # two-level column headers

    The dictionary form gives a MultiIndex for the columns (("unit_price", "mean")), which is awkward to work with. Named aggregation avoids it.

    Counting products

    ("product_id", "count") counts non-missing IDs. Since IDs are never missing, that equals the number of rows. For a count of distinct products in a table where they repeat, such as order lines, use ("product_id", "nunique") instead. Choosing between count, size and nunique is a correctness decision, not a style one.

    Reading the result

    Every category has 20 products, so the interesting columns are the prices. Furniture’s average is far higher than Decor’s; compare the average with the maximum to spot categories where one or two premium items pull the average up.

    Common mistakes

    • Writing n_products=("product_id", count) with the bare name count, which isn’t defined. Use the string "count" (or the built-in len).
    • Forgetting as_index=False, so category is the index and not a column.
    • Mistyping the column inside the tuple: ("price", "mean") raises KeyError.

    If you know SQL

    Named aggregation is the pandas version of SELECT category, COUNT(product_id) AS n_products, AVG(unit_price) AS avg_price ....