6.2 Using map and apply Correctly

Section 6.1 showed that ordinary arithmetic and conditions belong in vectorized expressions. Some transformations do need a lookup or custom function. The important question is not “Can apply do this?” but “What shape of information does this operation need?”

Choose an interface by input and output shape

Different APIs communicate different contracts:

Choose the narrowest contract that expresses the task. It is easier to reason about a lookup as a lookup than as an arbitrary row function.

For example, a dictionary mapping is ideal for controlled codes:

zone_names = {
    "N": "North",
    "S": "South",
    "W": "West",
}

mapped = orders["zone_code"].map(zone_names)
unknown = orders["zone_code"].notna() & mapped.isna()

A plain dictionary maps unknown keys to missing values. That behavior is valuable when the dictionary is a controlled vocabulary: inspect orders.loc[unknown, "zone_code"] instead of hiding new codes behind a plausible default.

When mapping with a function, na_action="ignore" keeps missing values away from the callable:

labels = orders["customer_name"].map(
    lambda value: value.strip().casefold(),
    na_action="ignore",
)

Prefer vectorized string methods for that particular example—orders["customer_name"].str.strip().str.casefold()—because they state the domain more directly. map is most compelling for actual mappings or scalar logic without a clearer specialized operation.

Understand the axis before using DataFrame.apply

DataFrame.apply passes one Series to a function at a time:

• axis=0, the default, passes each column. The Series index contains row labels.

• axis=1 passes each row. The Series index contains column labels.

A custom row rule can therefore inspect several fields from the same observation:

def shipping_fee(row):
    if pd.isna(row["weight_kg"]):
        return pd.NA

    surcharge = 8.0 if row["zone"] == "remote" else 0.0
    return 5.0 + 1.2 * row["weight_kg"] + surcharge

fee = shipments.apply(shipping_fee, axis=1).astype("Float64")

Without axis=1, the function receives a whole column and row["weight_kg"] does not mean what its name suggests. During debugging, call the function on shipments.iloc[0] or temporarily inspect the Series it receives.

The function's return value affects output shape. A scalar per row produces a Series. Returning a named Series can expand into columns:

def fee_details(row):
    base = 5.0 + 1.2 * row["weight_kg"]
    surcharge = 8.0 if row["zone"] == "remote" else 0.0
    return pd.Series({"base_fee": base, "surcharge": surcharge})

details = shipments.apply(fee_details, axis=1)

Returning lists of inconsistent length or mixed types makes the output harder to predict. Define a stable return schema. A function passed to apply must also not mutate the row or column object it receives; mutation inside a user-defined function is unsupported and can cause surprising behavior.

Row-wise apply remains a Python-level loop in the ordinary case. After establishing a correct reference implementation, look for a vectorized equivalent:

fee_vectorized = (
    5.0
    + 1.2 * shipments["weight_kg"].astype("Float64")
    + shipments["zone"].eq("remote").mul(8.0)
).where(shipments["weight_kg"].notna())

pd.testing.assert_series_equal(
    fee,
    fee_vectorized,
    check_names=False,
)

Benchmark representative data only after verifying equivalent missingness, dtype, index, and values. A faster expression that implements a different policy is not an optimization.

Recognize aggregation and transformation

Use agg when many input rows become fewer summary values:

summary = orders[["amount", "delay_minutes"]].agg(["min", "median", "max"])

Use transform when a group calculation must return one aligned value per original row:

zone_median = orders.groupby("zone")["amount"].transform("median")
orders["amount_vs_zone"] = orders["amount"] - zone_median

The output-shape test is a reliable guide: reduction changes the number of observations; transformation preserves it. The next section applies these ideas to three common transformations whose parameters carry analytical meaning: bins, ranks, and category encodings.