4.2 Selecting Rows and Columns Safely

Chapter 3 used NumPy positions and Boolean masks. pandas adds labels, so every selection must answer two questions: are the coordinates labels or positions, and should the result remain two-dimensional? Making those choices explicit prevents code from silently changing meaning when rows are reordered or columns are added.

We will use this table:

import pandas as pd

deliveries = pd.DataFrame(
    {
        "zone": ["North", "South", "West", "Central", "North"],
        "minutes": [31, 52, 44, 38, 47],
        "fee": [8.5, 9.0, 7.5, 8.0, 10.0],
        "delivered": [True, True, True, False, True],
    },
    index=pd.Index(
        ["A104", "A105", "A106", "A107", "A108"],
        name="order_id",
    ),
)

Brackets select columns, but result shape depends on syntax

A single column label returns a Series:

minutes = deliveries["minutes"]
print(minutes.ndim)  # 1

A list of column labels returns a DataFrame, even when the list contains one label:

one_column_table = deliveries[["minutes"]]
print(one_column_table.ndim)  # 2

summary = deliveries[["zone", "minutes"]]

This distinction affects downstream operations. A function expecting a DataFrame may rely on .columns, while a Series has .name instead. Choose the result contract rather than adding or removing brackets by trial and error.

Avoid attribute-style access such as deliveries.zone in durable code. It fails for column names containing spaces, conflicts with DataFrame method names, and cannot express a dynamic column variable. deliveries["zone"] is unambiguous.

.loc selects labels; .iloc selects positions

.loc receives row labels and column labels:

by_label = deliveries.loc[
    "A105":"A107",
    ["zone", "minutes"],
]

The label slice normally includes both A105 and A107. It expresses a range in the current index ordering. If labels are unsorted or duplicated, range slicing can become surprising or invalid; prefer explicit label lists when the order contract is unclear.

.iloc receives zero-based integer positions:

by_position = deliveries.iloc[1:4, [0, 1]]

The result is the same three rows and two columns. Position slicing follows Python's half-open rule, so position 4 is excluded.

These selectors remain distinct when labels themselves are integers:

sample = pd.DataFrame(
    {"value": [10, 20, 30]},
    index=[100, 1, 200],
)

sample.loc[1]   # row whose label is 1
sample.iloc[1]  # row currently at position 1

Those happen to identify the same row here, but reordering the index can separate them. Never use .loc when you mean “the second row,” or .iloc when you mean “order 1.”

For one scalar, .at[row_label, column_label] and .iat[row_position, column_position] are focused alternatives. Use them when the intent is genuinely scalar access, not as a premature performance trick.

The selector lab is an executable coordinate console. Write a .loc or .iloc slice, run it, and then reorder the physical rows without changing the command. The highlighted source and returned DataFrame make inclusive label stops, exclusive positional stops, and the consequences of choosing the wrong coordinate system directly testable.

Boolean selection keeps rows where the condition is true

A comparison on a Series produces a Boolean Series carrying the same index:

slow = deliveries["minutes"] > 40
late_orders = deliveries.loc[slow]

The index is part of the mask. pandas aligns a Boolean Series to the table's row labels, protecting against accidental reordering:

reversed_mask = slow.sort_index(ascending=False)
same_rows = deliveries.loc[reversed_mask]

Although the mask order changed, its labels still connect each truth value to the correct order. An unalignable Boolean Series raises an error instead of being silently used by position.

A NumPy Boolean array has no labels, so pandas must apply it positionally. Use that only when positional coupling is intentional and lengths have been checked. In labeled analysis, preserving the Series index is usually safer.

Several helpers create readable conditions:

target_zone = deliveries["zone"].isin(["North", "South"])
ordinary_duration = deliveries["minutes"].between(30, 50)
complete = deliveries["minutes"].notna()

selected = deliveries.loc[
    target_zone & ordinary_duration & complete,
    ["zone", "minutes"],
]

Use &, |, and ~ for element-wise Boolean operations, and parenthesize comparisons. Python's and, or, and not expect one truth value and cannot decide the truth of an entire Series.

Nullable Boolean conditions can contain pd.NA. When used for indexing, missing mask entries are treated as false. That behavior may exclude records whose eligibility is unknown. Decide whether to reject, inspect, or explicitly fill missing conditions before selecting.

Strict selection should validate its contract

Selecting a missing label with .loc raises KeyError:

deliveries.loc[["A104", "A999"]]

That failure is valuable when every requested order must exist. If partial results are acceptable, make the policy explicit instead of catching every KeyError:

requested = pd.Index(["A104", "A999"])
missing = requested.difference(deliveries.index)

if len(missing):
    print("Missing orders:", missing.tolist())

partial = deliveries.reindex(requested)

reindex constructs the requested label set and inserts missing rows. It expresses “preserve this requested layout, even when data is absent,” which is different from strict .loc selection.

For reusable analysis functions, define and test:

• Whether row and column labels must exist.

• Whether the row index must be unique.

• Whether order follows the source or the caller's requested labels.

• Whether a one-column result is a Series or DataFrame.

• Whether an empty selection is valid or an error.

The contract lab acts as a small API fuzzer. Write the source index, a collection of normal and adversarial requests, and policies for missing labels, duplicates, order, dimensionality, and empty input. Every edit reruns the calls, exposing whether the contract returns, reindexes, reorders, or raises in each case.

Selection should answer a question without changing the source. The next section composes filters, establishes deterministic ordering, creates derived columns, and updates chosen cells without ambiguous chained assignment.