15.4 Inline Caches and Runtime Profiling

Dynamic operations are expensive when implemented as fresh general-purpose lookups every time. Accessing obj.x may require checking the receiver type, walking prototype or class metadata, resolving descriptors, and handling user-defined hooks. Calling f(a) may need to identify f, validate arity, and choose a dispatch rule. Yet each source site often sees only a few receiver shapes or call targets. An inline cache (IC) remembers those local observations.

The cache is “inline” because its fast-path data or code is associated with one operation site, not because it necessarily performs source-level function inlining. A bytecode interpreter can attach cache entries to decoded instructions. A JIT can embed guards and direct loads/calls in generated machine code. Both retain a generic miss path that performs full semantics and updates the cache.

Object shapes make property layout testable

Many dynamic-language VMs assign an object a shape, hidden class, map, or structure describing its property names, offsets, and attributes. Objects created by the same sequence of additions can share one shape. A property-load IC can test receiver.shape == Shape42; on success it loads from the cached offset. A mutation that adds, deletes, or reconfigures a property transitions the object to another shape.

A site seeing one shape is monomorphic. It can use one guard and one offset. A polymorphic inline cache (PIC) stores a short chain or table for several shapes. Beyond a configured limit, the site becomes megamorphic and switches to a shared lookup structure rather than growing code indefinitely. Monomorphic is not a correctness requirement; it is an observed state that can change as the program runs.

Prototype mutation, proxy behavior, dictionary-mode objects, getters, and access controls complicate caching. The cached result must include every assumption that made the lookup valid. A dependency or version guard can invalidate entries when prototype structure changes. Miss handling must be atomic enough for concurrent execution and garbage-collector movement.

Profiles turn execution history into optimization evidence

Runtime profiling collects information such as function and loop counts, branch frequencies, call targets, receiver shapes, value types, allocation sites, and exception rates. Instrumentation counters are precise for events they record but execute extra operations on hot paths. Sampling interrupts execution periodically and attributes observations to PCs or stack traces, reducing overhead but introducing statistical error. Hybrid systems use cheap counters to find candidates and detailed instrumentation only where useful.

Profiles guide several optimizations. Block layout places the frequent successor as fallthrough. A call site dominated by one target becomes an inlining candidate. Stable integer types permit unboxed arithmetic under guards. Allocation profiles can drive escape analysis or pretenuring. The profile is evidence, not proof: optimized code still needs guards or validity dependencies when language semantics permit other behavior.

Confidence, phase changes, and feedback stability

Small sample counts can be misleading. Seeing one target twice does not establish durable monomorphism. A runtime may require a minimum count and dominance ratio before specializing. It can use hysteresis so a site does not repeatedly alternate between two versions near a threshold. Compilation budgets rank candidates by expected benefit rather than compiling every profile anomaly.

Programs also change phase. Startup initializes modules with varied types, then a stable request loop dominates. A game alternates menus, gameplay, and loading. A server deploy changes traffic shape. Profiles need aging, epoching, or windowing so ancient observations do not permanently poison a site, while avoiding overreaction to one rare event. Deoptimization feedback can lower confidence or blacklist an unstable optimization temporarily.

Ahead-of-time profile-guided optimization has a similar risk: training input may not represent production. Profile identifiers must match the exact binary or IR layout; otherwise counters attach to the wrong functions or edges. Toolchains commonly merge profiles across runs, normalize counts, preserve context for inlining, and warn when data is stale or mismatched.

Test the feedback system as a control loop. Feed stable, alternating, and abruptly changing distributions; verify IC transitions and eviction; force GC while cache entries reference metadata; test invalidation after prototype or method changes; compare sampled estimates with known counts; and measure observer overhead. A fast path is valuable only if its miss path, invalidation, and recovery remain correct under the cases that made the fast path fail.