1. Lighthouse

Level 5 · Count orders by status

The request

From Megan Collins, Head of Sales

“How many of our orders were actually completed, and how many were cancelled, refunded or still pending? I want the whole breakdown.”

Your goal

Set result to a Series with the number of orders per status, as returned by value_counts(): index = status values, sorted from most to least frequent.

  • Use the orders DataFramenot done yet
  • Show exactly status countnot done yet
  • 4 rows, sorted as askednot done yet

Your code

Python

Loading the editor…

    ResultPreview of orders · press Run to see yours
    Preview of orders
    order_idcustomer_idorder_datestatuschannel
    10011252023-01-01 20:51:19completedweb
    10021562023-01-02 07:58:46completedweb
    1003222023-01-02 10:25:27completedmarketplace
    1004482023-01-02 11:21:51completedmarketplace

    Mote checks your result

    Press Run. Each row that matches the goal becomes a firefly in your lantern.

    DataFrames

    Loaded for you as pandas DataFrames, plus pd and np.

    orders12,000 rows · 7 columns
    • order_idinteger
    • customer_idinteger
    • order_datetimestamp
    • statusvarchar
    • channelvarchar
    • shipping_feedecimal
    • coupon_codevarchar
    First 5 rows of orders
    order_idcustomer_idorder_datestatuschannelshipping_feecoupon_code
    10011252023-01-01 20:51:19completedweb0.00NaN
    10021562023-01-02 07:58:46completedweb0.00NaN
    1003222023-01-02 10:25:27completedmarketplace4.99WELCOME10
    1004482023-01-02 11:21:51completedmarketplace4.99WELCOME10
    1005822023-01-02 11:43:36completedweb0.00WELCOME10
    customers2,018 rows · 9 columns
    • customer_idinteger
    • first_namevarchar
    • last_namevarchar
    • emailvarchar
    • cityvarchar
    • statevarchar
    • signup_datedate
    • segmentvarchar
    • referral_sourcevarchar
    First 5 rows of customers
    customer_idfirst_namelast_nameemailcitystatesignup_datesegmentreferral_source
    1OwenHudsonowen.hudson@icloud.comSan FranciscoCA2022-06-02consumersearch
    2DouglasMendozadmendoza69@gmail.comAtlantaGA2022-06-02consumerNaN
    3HenryBrookshbrooks@lakesidecafe.comLos AngelesCA2022-06-04businessNaN
    4BrianGrahambgraham@bluecedarhomes.comSeattleWA2022-06-06businesspinterest
    5KyleJimenezkjimenez27@yahoo.comIndianapolisIN2022-06-07consumerNaN
    products120 rows · 8 columns
    • product_idinteger
    • namevarchar
    • categoryvarchar
    • subcategoryvarchar
    • unit_pricedecimal
    • unit_costdecimal
    • launched_ondate
    • is_activeboolean
    First 5 rows of products
    product_idnamecategorysubcategoryunit_priceunit_costlaunched_onis_active
    101Vale Velvet LoveseatFurnitureSofas1659.00NaN2023-09-11true
    102Alder Oak Coffee TableFurnitureTables919.00503.502024-05-16true
    103Orchard Rattan Dining ChairFurnitureChairs789.00429.872023-05-27true
    104Upland Oak BookcaseFurnitureStorage429.00214.702022-12-20true
    105Avery Leather Bed FrameFurnitureBeds449.00232.702021-08-31true
    order_items30,338 rows · 6 columns
    • order_item_idinteger
    • order_idinteger
    • product_idinteger
    • quantityinteger
    • unit_pricedecimal
    • discount_pctdecimal
    First 5 rows of order_items
    order_item_idorder_idproduct_idquantityunit_pricediscount_pct
    11001121931.990.00
    21001154311.990.00
    31002154411.990.00
    410021941149.000.00
    51002193219.990.00
    returns1,673 rows · 5 columns
    • return_idinteger
    • order_item_idinteger
    • returned_ondate
    • reasonvarchar
    • refund_amountdecimal
    First 5 rows of returns
    return_idorder_item_idreturned_onreasonrefund_amount
    1142023-01-10damaged83.69
    2422023-01-10wrong_item19.99
    31052023-01-14wrong_item10.99
    4242023-01-16changed_mind26.99
    51262023-01-16late20.99
    employees60 rows · 8 columns
    • employee_idinteger
    • first_namevarchar
    • last_namevarchar
    • departmentvarchar
    • titlevarchar
    • manager_idinteger
    • hire_datedate
    • salaryinteger
    First 5 rows of employees
    employee_idfirst_namelast_namedepartmenttitlemanager_idhire_datesalary
    1CatherineHaleExecutiveChief Executive OfficerNaN2016-03-01285000
    2RobertKimExecutiveChief Operating Officer12017-01-09235000
    3MeganCollinsSalesHead of Sales12018-04-16172000
    4ChloeGutierrezSalesAccount Executive32020-01-2172000
    5FrankBellSalesAccount Executive32023-05-2173000
    web_sessions40,000 rows · 7 columns
    • session_idinteger
    • customer_idinteger
    • started_attimestamp
    • devicevarchar
    • sourcevarchar
    • pages_viewedinteger
    • convertedboolean
    First 5 rows of web_sessions
    session_idcustomer_idstarted_atdevicesourcepages_viewedconverted
    1NaN2023-01-01 23:02:24mobilepaid1false
    2182023-01-01 15:07:49desktoppaid1false
    3NaN2023-01-01 16:55:46desktopdirect7false
    4NaN2023-01-01 12:43:28mobileorganic3false
    5NaN2023-01-01 13:06:21mobilesocial2false
    marketing_spend3,276 rows · 3 columns
    • spend_datedate
    • channelvarchar
    • amountdecimal
    First 5 rows of marketing_spend
    spend_datechannelamount
    2023-01-01paid492.99
    2023-01-01email37.38
    2023-01-01social200.53
    2023-01-02paid709.13
    2023-01-02email64.13

    The lesson

    value_counts: how many of each?

    For a text or category column, the first question is always “what values are there, and how often?”. value_counts() answers it in one call:

    orders["status"].value_counts()
    status
    completed    10849
    cancelled      726
    refunded       304
    pending        121
    Name: count, dtype: int64

    The result is a Series: the values become the index and the counts are the data. It’s sorted from most to least frequent, so the top of the list is the answer to “what’s most common?”.

    Useful options:

    orders["status"].value_counts(normalize=True)   # shares that add up to 1
    orders["coupon_code"].value_counts(dropna=False)   # include NaN as its own row
    orders["channel"].value_counts(ascending=True)     # least common first

    The dropna=False option is important. By default, missing values are silently left out, and for coupon_code the missing rows (no coupon used) are by far the largest group. A share computed without them would be badly wrong.

    orders["channel"].unique()     # array(['web', 'marketplace', 'app'], dtype=object)
    orders["channel"].nunique()    # 3
    orders["customer_id"].nunique()   # how many distinct customers ordered

    nunique is how you answer “how many different customers?”, which is not the same as the number of orders.

    describe: numbers at a glance

    For numeric columns, describe() gives the standard summary:

    products["unit_price"].describe()
    count     120.00
    mean      270.16
    std       362.06
    min        10.99
    25%        51.74
    50%       154.00
    75%       289.00
    max      1659.00

    Read it like an analyst. The mean ($270) is far above the median, the 50% line ($154). That gap tells you prices are skewed: a few expensive pieces of furniture pull the average up. For skewed data, the median is usually the better “typical” value to report. The quartiles (25% and 75%) tell you the middle half of products cost between about $52 and $289.

    On a whole DataFrame, orders.describe() summarises every numeric column, and orders.describe(include="object") summarises text columns with count, unique, top and freq.

    Watch for IDs

    describe() will happily report the mean customer_id. It’s meaningless: IDs are labels, not quantities. Ignore summaries of ID columns, or select the columns you actually care about first.

    Common mistakes

    • Calling value_counts() on the whole DataFrame when you meant one column. orders.value_counts() counts unique combinations of every column.
    • Forgetting dropna=False on columns with missing values, then reporting shares that are too high.
    • Reporting the mean of a skewed column as “the typical price”.

    If you know SQL

    orders["status"].value_counts() is SELECT status, COUNT(*) FROM orders GROUP BY status ORDER BY COUNT(*) DESC.