Roadmap
Thirty-six chapters across four topics. Pick a topic to see its path, or All for the whole map. Drag to explore, click a chapter to jump in.
Thirty-six chapters across four topics. Pick a topic to see its path, or All for the whole map. Drag to explore, click a chapter to jump in.
Bytes & Addresses
Physics of Storage
Latency vs Bandwidth
The Memory Wall
Caches
Locality
Data Layout
Virtual Memory
CPUs Fight Latency
Capstone: Napkin Math
Two Philosophies
The Throughput Machine
GPU Memory Hierarchy
Host ↔ Device
Roofline
Capstone: Spec Sheet
Kernels, Threads, Blocks
Warps & Divergence
Occupancy
Coalescing
Cooperation
Bank Conflicts
The GEMM Arc
Streams, Async, Fusion
Precision
Capstone: GEMM
Transformer Memory Traffic
Prefill vs Decode
KV Cache
FlashAttention
Speculative Decoding
Batching
Paged KV Cache
Quantization
Scaling Out
Capstone: Tokens/sec