11.1 Scaling, Distance, and Clustering

Chapters 9 and 10 learned from targets: delivery minutes for regression and late/on-time labels for classification. This section removes the target. We ask whether orders form useful groups from their feature geometry alone. That is unsupervised learning.

Unsupervised does not mean objective or assumption-free. No label tells the algorithm what “similar” should mean. Feature selection, scaling, distance, and the intended use define the answer before any cluster receives a color.

Design the distance before finding groups

For two rows $x$ and $z$ with $p$ numeric features, Euclidean distance is

Each coordinate difference is squared, so a large numeric range can dominate. If basket value spans 0–10,000 yuan while route distance spans 0–20 kilometers, basket value can decide nearly every neighbor. Converting yuan to cents multiplies that coordinate by 100 and can change the result even though the business facts are identical.

Standardization replaces each training value by

where $\mu_j$ and $s_j$ come only from the training or development data used to fit the analysis. New rows reuse those stored values through transform. Unit conversion then cancels, but the geometry is not automatically correct. StandardScaler is sensitive to outliers, and equal variance is not the same as equal business importance.

The distance lab shows the entire calculation. A purple diamond is a new order; lines connect it to historical orders. Switch revenue from yuan to cents with raw distance and watch the nearest neighbor change. Turn standardization on and the unit change disappears. Then alter a declared business weight and see that weighting is a modeling decision, not hidden preprocessing.

Distance also requires semantic compatibility. Euclidean distance assumes continuous coordinates where differences are meaningful. One-hot encoded categories, binary flags, heavy-tailed counts, circular hours, and missingness indicators create different geometries. An arbitrary category code such as bike=0, car=1, van=2 invents order and equal spacing. Select or transform features because they express the similarity question you care about.

In high dimensions, many pairwise distances become less distinguishable. Redundant correlated columns can count one concept several times. Inspect distributions, correlations, outliers, and distance contributions before interpreting any cluster map.

Watch K-means alternate assignment and update

K-means asks for $k$ clusters represented by centroids $\mu_1,\ldots,\mu_k$. It minimizes within-cluster squared distance, called inertia:

One run alternates two steps:

1. Assign each point to its nearest centroid.

2. Replace each centroid with the mean of its assigned points.

The steps repeat until assignments or centroids stabilize. The objective can settle at a local solution, so initialization matters. k-means++ spreads initial centers intelligently; repeated initializations provide additional protection. Use an explicit n_init and random_state when reproducibility matters.

from sklearn.cluster import KMeans
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

clusterer = make_pipeline(
    StandardScaler(),
    KMeans(n_clusters=3, n_init=20, random_state=42),
)

cluster_labels = clusterer.fit_predict(development[features])

Cluster IDs are arbitrary. Another fit may call the same geometric group 0 instead of 2. An ID is neither a score nor a hidden true class. Attach it to a copy of the original data, then profile sizes, medians, ranges, missingness, time periods, and operational outcomes not used to create the clusters.

The partition lab gives you a blank plane rather than curated examples. Draw compact blobs, curved moons, bridges, outliers, or uneven densities; plant the initial centroids yourself; then execute one mean update or run to convergence. The computed Voronoi regions and inertia expose which structures K-means can express, how initialization changes the local solution, and when an empty or tiny cluster appears.

Choosing $k$ needs several kinds of evidence. Inertia always decreases as $k$ increases, so seek a useful elbow rather than its minimum. The silhouette coefficient compares a point's mean distance $a$ to its own cluster with the nearest other-cluster distance $b$:

Values near 1 indicate separation, values near 0 indicate a boundary, and negative values suggest a point may fit another cluster better. A mean silhouette can still hide a tiny or unstable group. Compare multiple seeds or bootstrap samples with a label-permutation-invariant measure such as adjusted Rand index, and inspect every cluster's size.

K-means is one model of grouping, not the definition of clustering. DBSCAN and related density methods can discover irregular connected regions and mark sparse points as noise, but they introduce neighborhood scale and density assumptions. Hierarchical clustering exposes nested merges, but linkage and cut height change the result. Compare algorithms only after stating the geometry and business use.

A cluster is useful when it is reproducible enough, understandable in original units, actionable without protected-variable harm, and better than a simpler segmentation rule. The next section uses principal component analysis to rotate and compress high-dimensional geometry before we visualize or model it.