3.4 Performance, Memory, Views, and Copies

NumPy's compact syntax can hide an important distinction: two arrays may display the same values while having different ownership relationships. One may own its data, another may be a view into that data, and a third may hold an independent copy. Correctness comes first, because an unexpected shared buffer can make an innocent assignment alter data elsewhere.

An ndarray is data plus metadata

Conceptually, an ndarray combines a raw data buffer with metadata such as:

• dtype, which determines the size and interpretation of each element.

• shape, which gives the length of every axis.

• strides, which give the byte movement needed to advance one position along each axis.

• Ownership information and an optional reference to a base array.

import numpy as np

a = np.arange(12, dtype=np.int64).reshape(3, 4)

print(a.shape)     # (3, 4)
print(a.strides)   # commonly (32, 8): 4 * 8 bytes, then 8 bytes
print(a.itemsize)  # 8 bytes per element
print(a.nbytes)    # 96 bytes of array payload

nbytes is approximately size * itemsize. It counts the element payload, not every Python-object, allocator, or process overhead.

A view has new array metadata but refers to storage owned elsewhere. A copy owns a separate buffer. Basic slicing usually creates a view:

a = np.array([10, 20, 30, 40, 50, 60])
window = a[1:4]
window[0] = 99

print(a)  # [10 99 30 40 50 60]

The assignment changed a because window shares its buffer. This behavior makes slices efficient, but it also creates an ownership contract that the code should make clear.

Integer-array and Boolean advanced indexing normally create copies:

a = np.array([10, 20, 30, 40, 50, 60])
picked = a[[1, 3, 5]]
picked[0] = 99

print(a)  # unchanged

An explicit .copy() communicates that downstream code may modify its result independently:

owned = a[1:4].copy()

Use np.shares_memory to ask whether two arrays definitely overlap:

print(np.shares_memory(a, a[1:4]))       # True
print(np.shares_memory(a, a[[1, 3, 5]])) # False

.base can be informative, but it is not a complete public test of every ownership chain. shares_memory expresses the question directly.

Reshape and transpose may return views

reshape, ravel, and transpose often create views when the requested indexing pattern can be described with new shape and stride metadata. A transpose usually swaps strides without moving values:

a = np.arange(12).reshape(3, 4)
columns_first = a.T

print(columns_first.shape)    # (4, 3)
print(columns_first.strides)  # axes now step through memory differently

The transposed array is commonly non-contiguous in C order. Some later operations can consume that layout directly; others may make a contiguous copy. Whether reshape can return a view depends on the current strides and requested order, so never base correctness on an assumption that it always shares or always copies.

When an API accepts an array, document its mutation policy:

• Borrowed read-only: the function reads the input and must not modify it.

• Borrowed mutable: the caller permits documented changes to the input.

• Owned result: the function returns independent storage for callers to modify.

This vocabulary prevents defensive copying everywhere while keeping mutation intentional.

Vectorization moves work, but expressions can allocate

Chapter 2 showed that vectorized NumPy operations move iteration from Python into compiled loops. That often improves speed, but a compact expression may create large temporary arrays:

standardized = (x - x.mean()) / x.std()

Ignoring scalar reductions and implementation details, this may involve a full-size result for x - mean and another full-size result for division. For ten million float64 values, one payload is

Several live buffers can therefore create significant peak memory and memory traffic even when the expression is only one line.

Universal functions support an out parameter that can reuse a buffer:

mean = x.mean()
std = x.std()

result = np.empty_like(x)
np.subtract(x, mean, out=result)
np.divide(result, std, out=result)

This preserves x while reusing result. If the caller explicitly allows input mutation, in-place operations use less storage:

x -= mean
x /= std

In-place code is not automatically better. It destroys the original values, may fail for incompatible dtype casts, and can surprise other views sharing the same buffer. Treat mutation as an interface decision, not a clever local optimization.

Measure the real bottleneck

Optimization should follow evidence. Begin with representative data and preserve a correct reference implementation:

reference = (x - x.mean()) / x.std()

mean = x.mean()
std = x.std()
optimized = np.empty_like(x)
np.subtract(x, mean, out=optimized)
np.divide(optimized, std, out=optimized)

assert np.allclose(reference, optimized)

Then measure dimensions that matter to the workload:

• Wall-clock time over repeated runs, with setup separated from the measured statement.

• Peak memory, not only the final array's nbytes.

• Allocation count and full-array passes when memory bandwidth is limiting.

• Accuracy, including tolerance, missing values, overflow, and dtype changes.

Use timeit for focused timing and a memory profiler for allocation behavior. Warm-up, caching, hardware, NumPy version, linked numerical libraries, and input shape can all affect results. A tiny example does not prove that the same method wins at production scale.

If the entire dataset cannot fit comfortably in memory, process chunks whose boundaries preserve the calculation's meaning. Independent element-wise transforms can be chunked easily. Global statistics require a plan: compute stable aggregate state first, then transform chunks, or use a numerically sound streaming algorithm.

Avoid these common optimization traps:

• Replacing clear vectorized code with a Python loop based only on a guess.

• Converting to float32 without checking range and precision requirements.

• Using np.vectorize expecting compiled-loop performance; it is primarily a convenience wrapper around Python calls.

• Benchmarking once on an unrealistically small input.

• Saving memory by mutating an array whose ownership is shared or unknown.

Chapter 3 field checklist

Before trusting a practical NumPy pipeline, ask:

• Does every axis retain a documented business meaning after reshape, transpose, stack, or split?

• Do sorting permutations move all aligned arrays together, with explicit boundary and tie rules?

• Can every stochastic result be audited from generator context and sampling design?

• Is mutation intentional, and have views and copies been tested where ownership matters?

• Has performance been measured on representative data while numerical equivalence is checked?

These questions connect correctness, reproducibility, and efficiency. In the next chapter, pandas will attach labels to rows and columns, but it will not remove the need to reason about alignment, missingness, ownership, and evidence.