Hardware Map

Click a box to see what it is and where it is covered. Pick a path to step through how instructions and data move. Blue labels open the posts.

Hardware map: the layers one memory access passes through Blue = where it is covered (#N = Trading System Notes #N; MESI = the MESI topic post; Ref N.N = a section of the Reference page). Numbers are rough magnitudes. Solid arrows = how instructions and data flow; red dashed = flush and refetch after a branch mispredict; grey dashed = coherence messages between cores. CPU socket 0 = one NUMA node #1 pinning / isolation · NUMA Core 0 2 hardware threads share this core (SMT, #1) Front end: fetch instructions Back end: out-of-order execution Memory: loads and stores Branch predictor (BPU) BTB · RAS · direction predictor guesses where the next instruction is #7 branch prediction Fetch reads code from L1I at the predicted address; a wrong path is thrown away #7 speculative execution Decode instructions → micro-ops (µops) L1I 32 KiB (instruction cache) holds code, not data #6 function grouping Rename + ROB ROB = reorder buffer: instructions not yet retired Ref 2.3 ROB and MLP · #16 PAUSE Schedule → execution units ALU math · AGU addresses branches are resolved here #2 dependency chains · #12 the divider Retire commits results in program order Register file #2 register spills #6 storage classes dTLB (data TLB, translation) miss: walk 4 levels from the root in CR3 #6 2 MiB pages · #3 TLB shootdown Ref 2.1 TLB reach Store buffer stores queue here, then drain to L1D in order #2 store buffer #2 store-to-load forwarding Invalidate queue invalidations from other cores wait here #2 invalidate queue #2 acquire L1D 32 KiB · 8-way (data cache) LFB (line fill buffer) #6 alignment · sets Ref 2.3 LFB L2 512 KiB · 8-way (private to the core, code and data) #6 critical stride 64 KiB · #2 private L1/L2 Caches and memory move data in whole 64 B cache lines #2 · #6 how many lines one access touches µops commit store Core 1 … Core 5 (same structure as core 0) Core 1 Front Back L1I · L1D L2 Core 2 Front Back L1I · L1D L2 Core 3 Front Back L1I · L1D L2 Core 4 Front Back L1I · L1D L2 Core 5 Front Back L1I · L1D L2 MESI: one writer or many readers One cache line can have a copy in several cores' L1/L2. To write it, a core must first invalidate the others. MESI topic post #2 false sharing (two cores fight over a line) #4 cached cursor · #5 more readers, costlier writes #6 alignas(64) gives an object its own line #16 spinlocks: TTAS, queue locks, umwait Every core has its own L1 and L2 (and its own TLB and store buffer) another core wants to write a line: invalidations land in core 0's invalidate queue MESI L3 16 MiB · 16-way (shared by all cores) #2 shared L3 · #6 hashed set selection Memory controller: spreads consecutive addresses across channels (interleaving) #6 channel balance (line count coprime with channel count) DIMMs (DRAM) #6 Channel 0 DIMM → rank → many banks A bank = a table of rows × columns Row buffer: the row now open hit the open row ~15 ns switch rows (conflict) ~40 ns refresh every ~7.8 µs stalls 300–500 ns → P99 spikes Channel 1 same as channel 0 channels work in parallel: bandwidth ≈ channels × one channel Channel 2 … same as channel 0 Page tables (one per process) a 4-level tree; CR3 holds the root spread over channels like any data entries get cached in L1/L2/L3, so a TLB miss rarely comes this far #3 TLB shootdown CPU socket 1 its own cores, L3, controller and memory remote memory goes over the interconnect #1 NUMA: thread and memory on the same node Ref 2.1 first-touch placement socket interconnect NIC → interrupt → some core #1 steer IRQs away from hot cores (an OS layer, not a cache layer) The fields of one address (4 KiB pages, L1D 32 KiB 8-way, 64 B lines) Virtual page number (bits 12+) → TLB Huge pages: fewer page numbers, fit in the TLB Set index bits 6–11 #6 4 KiB multiples apart → same set Line offset bits 0–5 #6 aligned = these bits are 0 Page offset bits 0–11 don't change in translation, so L1 picks the set while the TLB translates (VIPT) → L1's sets × line size is capped at 4 KiB; it can only grow by adding ways Access latency magnitudes Cycles assume about 4 GHz. Each level down is roughly 3–10× slower. Access time Capacity Managed by ~1 cycle (< 1 ns) about 1 KB compiler Registers 4–5 cycles (~1 ns) 32–48 KiB / core hardware L1D cache ~12–20 cycles (~3–5 ns) 0.5–2 MiB / core hardware L2 cache ~40–120 cycles (~10–30 ns) about 10–500 MiB, shared hardware L3 cache ~300–500 cycles (~70–120 ns) about 16 GB to 4 TB OS (page tables) Local DRAM ~500–800 cycles (~120–200 ns) another socket's memory OS + where you pin Remote NUMA 10–100 µs TB scale OS (file system) NVMe SSD in core on chip off chip devices Numbers are magnitudes for intuition, not any specific machine. The map grows with the series: new hardware layers get labels as later posts cover them.