4.1 Series, DataFrame, Index, and Schema

NumPy arrays gave us shapes, numbered axes, and one dtype per array. Real analytical tables add another requirement: values need labels that preserve what each row and column means. pandas builds on NumPy's array model by attaching labels and allowing each column to use a dtype suited to its field.

Import pandas using its conventional alias:

import pandas as pd

Series: one labeled dimension

A pandas Series is a one-dimensional sequence of values paired with an Index:

minutes = pd.Series(
    [31, 52, 44],
    index=["A104", "A105", "A106"],
    name="delivery_minutes",
    dtype="Int64",
)

The values are delivery durations. The index labels are order IDs, and the Series name describes the measured variable. These three roles must not be confused: A104 identifies an observation, while 31 is a quantity on which arithmetic is meaningful.

Inspect the structure instead of relying on notebook display:

print(minutes.index)
print(minutes.name)
print(minutes.dtype)
print(minutes.shape)  # (3,)

Int64 with a capital I is pandas' nullable integer dtype. It can represent whole numbers together with pd.NA without converting identifiers or counts to floating-point values. Because dtype inference may differ by constructor and pandas version, declare important boundary types explicitly.

DataFrame: labeled rows and heterogeneous columns

A DataFrame is a two-dimensional labeled table. Unlike a two-dimensional NumPy array, its columns may have different dtypes:

order_id = pd.Index(
    ["A104", "A105", "A106", "A107"],
    name="order_id",
)

deliveries = pd.DataFrame(
    {
        "zone": pd.Series(
            ["North", "South", "West", "South"],
            index=order_id,
            dtype="string",
        ),
        "minutes": pd.Series(
            [31, 52, None, 44],
            index=order_id,
            dtype="Int64",
        ),
        "delivered": pd.Series(
            [True, True, False, True],
            index=order_id,
            dtype="boolean",
        ),
    },
    index=order_id,
)

The table has two labeled axes:

• Axis 0 is the row Index, containing order IDs.

• Axis 1 is the column Index, containing variable names.

Each column is also a Series aligned to the DataFrame's row index. That is why the constructor above supplies index=order_id to every Series. If a Series had a different index, pandas would align by labels rather than copying its values by position.

Useful first inspections include:

print(deliveries.shape)
print(deliveries.columns)
print(deliveries.index.name)
print(deliveries.dtypes)
print(deliveries.head())
print(deliveries.info())

head previews records; it does not validate the complete table. info summarizes non-missing counts, dtypes, and memory usage, but domain checks still belong in explicit assertions.

The schema lab gives you an editable table and an editable schema manifest. Change any cell, add records, move the index declaration, or alter nullability and dtypes. Validation runs over the whole table, so you can invent duplicate keys and type contamination that a pleasant-looking preview would miss.

An index is an alignment key, not a row number

The default RangeIndex uses labels 0, 1, 2, .... It is convenient when row identity is unimportant or already stored in a column. A business key such as order_id can become the index when it is stable, non-missing, and unique:

deliveries = raw.set_index("order_id", verify_integrity=True)

verify_integrity=True refuses duplicate keys during the operation. Alternatively, validate an existing index:

assert deliveries.index.notna().all()
assert deliveries.index.is_unique
assert deliveries.index.name == "order_id"

An index does not need to be unique in every pandas object. Duplicate labels are useful for some grouped or time-series structures. But if the contract says one row per order, duplicates are a data error: selecting one order label could unexpectedly return several rows.

Do not place a value in the index merely to make the display attractive. Index labels participate in alignment, selection, joins, and serialization. If a field is not a stable identity or coordinate, it usually belongs in a regular column.

Arithmetic aligns labels before values

Consider demand and capacity Series whose labels and order differ:

demand = pd.Series(
    {"North": 12, "South": 8, "West": 5},
    dtype="Int64",
)

capacity = pd.Series(
    {"South": 10, "North": 15, "Central": 4},
    dtype="Int64",
)

surplus = capacity - demand

pandas forms the union of labels, aligns both operands, and only then subtracts. North and South have both values. Central lacks demand, while West lacks capacity, so their surplus is missing.

This behavior prevents a positional mistake: the first capacity value belongs to South, while the first demand value belongs to North. A NumPy-style positional subtraction would combine unrelated zones.

You can inspect alignment directly:

aligned_capacity, aligned_demand = capacity.align(
    demand,
    join="outer",
)

When the domain contract says absence means zero, a named arithmetic method accepts fill_value:

surplus_assuming_zero = capacity.subtract(demand, fill_value=0)

That is a business assumption, not a harmless cleanup. Missing capacity may mean “not reported,” not zero capacity. Keep the missing result visible until the meaning is known.

The alignment lab lets you author both labeled Series as label:value records and execute an alignment expression. Reorder, rename, add, or delete labels and inspect the computed union. You can then make the missing-value policy explicit and see exactly when filling with zero changes a structural mismatch into a domain claim.

Treat schema as an executable contract

A schema describes the expected structure and meaning of a dataset: required columns, dtypes, keys, nullability, ranges, units, and sometimes relationships between fields. pandas does not provide one universal schema declaration, but ordinary assertions can protect a notebook or pipeline boundary:

required = {"zone", "minutes", "delivered"}

assert required <= set(deliveries.columns)
assert deliveries.index.is_unique
assert deliveries["zone"].dtype == pd.StringDtype()
assert str(deliveries["minutes"].dtype) == "Int64"
assert deliveries["minutes"].dropna().between(0, 24 * 60).all()

Use exact column-order checks only when order is part of an external file or API contract. For ordinary analysis, set inclusion is often more robust. Likewise, a dtype check proves storage type, not semantic validity: a negative duration still fits an integer dtype.

The next section turns these labeled structures into precise selection contracts. You will choose rows and columns by label, by position, and by Boolean condition without confusing one coordinate system for another.