6.3 Binning, Ranking, and Categorical Encoding

Numbers often need a second representation. A delivery duration may become a service band, a score may become a rank, and a sales channel may become model-ready indicator columns. These operations are not neutral formatting: their boundaries, tie policies, and category assumptions change what downstream analysis can see.

Turn continuous values into declared intervals

pd.cut assigns values to intervals defined by value boundaries:

import numpy as np
import pandas as pd

service_level = pd.cut(
    deliveries["minutes"],
    bins=[0, 15, 30, 60, np.inf],
    labels=["fast", "normal", "slow", "critical"],
    right=True,
    include_lowest=True,
    ordered=True,
)

With right=True, intervals are right-closed: a duration of 30 belongs to $(15,30]$. include_lowest=True makes the first lower boundary part of the first interval. Always test values exactly on every boundary; an off-by-one interval error can change operational decisions.

Use cut when boundaries come from a policy, scientific definition, or interpretable scale. Equal-width bins can also be requested with an integer, but their edges then depend on the observed range.

pd.qcut instead derives boundaries from sample quantiles:

quartile = pd.qcut(
    customers["annual_spend"],
    q=4,
    labels=["Q1", "Q2", "Q3", "Q4"],
)

Its goal is similar observation counts, not equal numeric widths. Boundaries change when the sample changes, so a Q4 label is relative to that fitted population. Repeated values can make requested quantile edges identical. Do not casually use duplicates="drop": it can return fewer bins than requested and invalidate a fixed label list. Inspect returned edges with retbins=True and validate the actual category count.

Missing input remains missing. Values outside explicit cut edges also become missing, which may mean the range contract is incomplete rather than that the observation was absent.

Make tie behavior part of a rank definition

Ranking requires a direction and a tie rule:

customers["spend_rank"] = customers["annual_spend"].rank(
    method="min",
    ascending=False,
)

For descending scores [92, 88, 88, 71], common methods produce different results:

first makes prior sorting part of the definition. If the source order is accidental, the rank is accidental too. For within-group ranks, group first:

customers["region_rank"] = (
    customers.groupby("region")["annual_spend"]
        .rank(method="dense", ascending=False)
)

Percentage ranks can help compare groups of different sizes, but their exact interpretation still depends on tie method and denominator. State the rule whenever rank drives eligibility or top-$k$ selection.

Encode category meaning, not just category text

A categorical variable has a limited set of allowed values. It may be nominal, with no order, or ordinal, with meaningful order.

For an ordinal priority field, declare the order explicitly:

from pandas.api.types import CategoricalDtype

priority_type = CategoricalDtype(
    categories=["low", "medium", "high"],
    ordered=True,
)
orders["priority"] = orders["priority"].astype(priority_type)

Sorting now follows the declared category order rather than alphabetical order. Any value outside the allowed categories becomes missing, so audit conversion failures before continuing.

For a nominal channel, integer codes such as web = 1, store = 2, partner = 3 invent an order and distance. Many models will treat 3 as larger than 1. One-hot encoding creates one membership column per category instead:

channel_columns = pd.get_dummies(
    customers["channel"],
    prefix="channel",
    dtype=bool,
)

Encoding for machine learning must be fitted on training data only. Save the training category schema, detect unseen categories in validation or production data, and align output columns deliberately. Otherwise, train and inference matrices can have different shapes—or an unknown category can be mistaken for the omitted baseline.

Bins, ranks, and encodings all compress or reinterpret evidence. Preserve the original value alongside the derived representation and version the policy when it matters. Section 6.4 changes a different kind of representation: it moves variables between columns and rows while aiming to preserve the underlying observations.