Memory Model and Data Layout on the JVM

Core Concepts

The Memory Wall

Before examining any Java-specific behavior, the physics must be established. Modern CPUs execute instructions in nanoseconds, but reaching main memory costs approximately 200 CPU cycles or 50–100 nanoseconds. That gap — between CPU speed and DRAM latency — is the memory wall problem. A server handling 100,000 requests per second can spend 70% of execution time simply waiting for memory if data layout does not cooperate.

The hardware response to the memory wall is a hierarchy of increasingly fast but smaller caches:

Level	Typical Size	Typical Latency
L1	32–64 KB	~1 ns
L2	256 KB – 2 MB	~5 ns
L3	4–64 MB	~10–20 ns
Main memory	GBs	~100 ns

When over 95% of data requests are served within 1–2 cycles, latency is hidden. When they are not, performance collapses.

Cache Lines: The Atomic Unit of Memory Transfer

The CPU does not fetch bytes; it fetches cache lines. On x86-64, a cache line is 64 bytes. When any byte is read, the entire 64-byte line surrounding it is pulled into cache. The implication is sharp: accessing the first byte of a cache line is essentially free — the hardware already fetched the rest. Accessing scattered random locations that don't share cache lines incurs the full ~200-cycle penalty each time.

Sequential and contiguous memory access patterns exploit this through spatial locality: if a program accesses one memory location, it will likely access nearby locations soon. The cache line mechanism turns that likelihood into a hardware guarantee. Temporal locality — accessing the same location repeatedly — is similarly exploited by keeping hot data resident in L1 and L2.

The JVM Object Header: A Hidden Tax

Every Java object — including Integer, Boolean, and a single-field DTO — carries a mandatory header on 64-bit JVMs:

• Mark word (8 bytes): stores the identity hash code, biased-locking state, and GC metadata (age bits, forwarding pointers during collection).
• Klass word (4 bytes): a compressed pointer to the class metadata structure.
• Padding (0–4 bytes): alignment to an 8-byte boundary.

The result is 12–16 bytes of overhead per object on typical 64-bit architectures. Arrays carry an additional 4 bytes for the length field.

This overhead is negligible for large objects. For small value-like objects it is ruinous. Consider an ArrayList<Integer>:

• Each Integer wraps a 4-byte int.
• Each Integer object carries a 16-byte header.
• The ArrayList stores references (8 bytes each on uncompressed OOPs).
• Per logical integer: 4 bytes of data, 24 bytes of overhead. Over 80% of memory is wasted.

The mark word's locking and GC metadata fields exist because every Java object is a first-class entity with identity. The language promised that any object could be synchronized on, could have its identity hash computed, and could be tracked through GC. The header is the price of that promise.

GC Pressure from Header Overhead

The header overhead compounds through garbage collection. GC heaps typically require 2–3x the size of live data to operate without constant collection cycles, since the collector needs headroom to move and compact objects. When objects are small and numerous, the absolute number of objects the GC must trace, mark, and potentially relocate is high, increasing both pause frequency and the cost of each pause.

Pointer Chasing: How Java Kills Spatial Locality

In idiomatic Java OOP, collections hold references. An ArrayList<Point> does not contain 2D coordinates — it contains references to Point objects scattered across the heap wherever the allocator placed them at construction time.

When iterating that list, each step:

1. Loads the reference from the array (one cache line).
2. Follows the reference to the Point object — potentially a cold cache line hundreds of megabytes away.
3. Reads x and y from the object.

Non-systematic object placement from dynamic memory allocation increases cache misses when objects accessed together are not placed together in memory. Java's reference model makes contiguous value layout structurally impossible without architectural workarounds.

Compare & Contrast

Value Types vs. Reference Types

Value types store data directly on the stack or embedded within containing objects, enabling contiguous memory layouts and better cache locality. Reference types store a pointer to a heap-allocated object, incurring indirection and per-object overhead.

Property	Value type (C# struct, Rust struct)	Java reference type
Per-object header	None	12–16 bytes
Heap allocation	Optional (can be stack/inline)	Always
Array layout	Contiguous data	Array of pointers
Cache behavior on iteration	Sequential, predictable	Pointer chase per element
GC tracing cost	None / minimal	Per object
Identity semantics	No (copied)	Yes

Java primarily uses reference types; C# provides both value and reference types as first-class primitives; Rust defaults to value semantics with explicit heap allocation via Box<T>.

AoS vs. SoA

Array of Structures (AoS) and Structure of Arrays (SoA) represent the two canonical choices for laying out a collection of records.

AoS (the Java default):

[x0, y0, vx0, vy0, health0, name0 | x1, y1, vx1, vy1, health1, name1 | ...]

SoA (must be constructed manually in Java):

xs:       [x0, x1, x2, ...]
ys:       [y0, y1, y2, ...]
vxs:      [vx0, vx1, vx2, ...]
healths:  [health0, health1, health2, ...]

AoS stores each object sequentially, providing good locality for sequential object traversal when all fields are accessed together. SoA stores each field contiguously across all entities, so a loop that touches only xs and ys loads only those arrays — every byte fetched is used.

Changing data layout from AoS to SoA can yield 40–60% performance improvement in real workloads and is a key motivation behind column-store databases and ECS game architectures.

Java vs. Languages with Layout Control

C and C++ preserve field declaration order, letting programmers minimize padding by managing field arrangement. Rust's default repr(Rust) permits the compiler to reorder fields to optimize layout automatically; repr(C) restores declaration order when interop demands it.

Java exposes none of this. The JVM may reorder fields internally (HotSpot groups fields by type for alignment), but the programmer cannot declare a packed or reordered layout. This is the root constraint that data-oriented design on the JVM must work around.

Worked Example

Rewriting a Physics Simulation from AoS to SoA

Consider a particle simulation with 1,000,000 particles. Each particle has a position, velocity, and a rarely-touched name string.

AoS layout (typical Java OOP):

class Particle {
    double x, y;       // position
    double vx, vy;     // velocity
    String name;       // cold: used only for debugging
}

List<Particle> particles = new ArrayList<>();

Each Particle object carries:

• 16-byte header
• 8+8 bytes position (x, y)
• 8+8 bytes velocity (vx, vy)
• 8-byte reference to String name

Total: ~56 bytes per object, plus a separate String heap allocation.

When the physics integration loop runs x += vx * dt, each step dereferences a Particle reference. The name field occupies 8 bytes in every cache line fetched for position data — useful data, wasted space.

SoA layout (data-oriented refactor):

class ParticleSystem {
    double[] xs, ys;    // position arrays — hot
    double[] vxs, vys;  // velocity arrays — hot
    String[] names;     // cold — separate array, not loaded during physics

    ParticleSystem(int n) {
        xs = new double[n]; ys = new double[n];
        vxs = new double[n]; vys = new double[n];
        names = new String[n];
    }

    void integrate(double dt) {
        for (int i = 0; i < xs.length; i++) {
            xs[i] += vxs[i] * dt;
            ys[i] += vys[i] * dt;
        }
    }
}

What changed:

1. No per-particle object header. double[] arrays store raw primitive doubles with a single 20-byte array header shared across all elements. There are no per-element headers.
2. Hot/cold field separation. names is in a separate array. The integration loop never touches it, so its cache lines are never fetched during the hot path.
3. Cache line utilization. A 64-byte cache line holds 8 double values. Accessing xs[0] fetches xs[0] through xs[7] simultaneously. The loop achieves near-perfect spatial locality.
4. GC pressure reduced. Instead of 1,000,000 Particle objects, the GC traces 5 arrays.

Hot/Cold Field Separation in an Entity Graph

Suppose you have a Customer class queried millions of times for account balance checks, but the customer's billingAddress and orderHistory are accessed only during checkout:

// AoS: cold fields pollute cache during hot-path reads
class Customer {
    long id;              // hot
    double balance;       // hot
    String name;          // warm
    Address billingAddress;  // cold
    List<Order> orderHistory;  // cold
}

Separating hot fields from cold fields ensures that cache lines fetched for inner-loop operations contain only data actually used:

// Hot record — stays cache-resident during balance checks
class CustomerCore {
    long id;
    double balance;
    String name;
}

// Cold record — only loaded at checkout
class CustomerDetail {
    long customerId;  // foreign key back to core
    Address billingAddress;
    List<Order> orderHistory;
}

The balance-check service works entirely within CustomerCore[]. The checkout service joins CustomerCore with CustomerDetail only when needed. The hot-path's working set shrinks; more useful records fit in L1/L2.

Active Exercise

Exercise A: Estimating Cache Behavior

You have a list of 500,000 Order objects. Each Order contains:

• long orderId (8 bytes)
• double amount (8 bytes)
• Instant createdAt (reference, ~32 bytes including header)
• String status (reference, ~40 bytes including header)
• List<LineItem> items (reference, varies)

A reporting job sums amount across all orders.

Questions:

1. Roughly how many cache misses does the AoS approach incur per order, assuming Order objects are heap-scattered?
2. If you refactored to SoA with a double[] amounts array, how many cache lines does the summation loop fetch for 500,000 elements? (Hint: 64 bytes / 8 bytes per double = 8 doubles per cache line.)
3. Which fields would you place in a "hot" partition, and which in a "cold" partition, for the reporting job vs. a fulfillment job that reads items?

Exercise B: Spotting False Sharing

Two threads update a counter each:

class Counters {
    volatile long counterA = 0;
    volatile long counterB = 0;
}

When one thread writes to counterA, the cache coherency protocol (MESI/MOESI) invalidates the entire 64-byte cache line on every other core, even though counterB was not touched. This is false sharing.

Task: Sketch a fix using padding. counterA and counterB must occupy different cache lines. How many padding bytes are needed between them, and why?

(Reference: fixing false sharing by separating conflicting fields to different cache lines can improve performance by 1.39x on average.)