The problem › Why isn't all memory fast?
The byte array is made of atoms
Chapter 0.1 gave you memory as one flat array of bytes. Now the uncomfortable question: if we can build memory that answers in a nanosecond, why is most of your 16 GB answering in ninety? Why not just build all of it fast?
The answer is not that hardware vendors are cheap. The circuit that makes a bit fast and the circuit that makes a bit dense are different circuits, and each bit of your byte array has to pick one. A desktop system carries no more than a few tens of megabytes of the fast kind against many gigabytes of the dense kind, a ratio of roughly a thousand, and that ratio is physics, not pricing strategy. Everything in this course, from caches and shared memory to HBM and the entire hierarchy, is downstream of this one trade-off, so it is worth seeing the actual transistors once.
The mechanism › Two ways to store a bit
SRAM holds its bit; DRAM hoards charge
SRAM (static RAM) stores each bit in a bistable six-transistor cell: four transistors form two cross-coupled inverters, each inverter's output feeding the other's input so the pair has exactly two stable states, one per bit value, and two more transistors gate access for reads and writes.CS:APP's image is an inverted pendulum: stable leaning fully left or fully right, unstable anywhere between. As long as power flows, the cell holds its value indefinitely and even recovers from electrical noise by falling back to the nearest stable state; raise the word line and the state is readable almost immediately, with no refresh ever needed. The price is six transistors of silicon per bit.
DRAM (dynamic RAM) stores a bit as charge on a capacitor guarded by one transistor. The capacitor is around 30 femtofarads; fully charged, it holds only a few tens of thousands of electrons. One-T, one-C is about as small as a bit can get, which is why DRAM is what you buy by the gigabyte. But a capacitor is not a latch; it is a leaky bucket. Even through a resistance of a couple of tera-ohms, those few thousand electrons drain away: a cell loses its charge within roughly 10–100 ms. So every row must be periodically read and rewritten, a process called refresh, and the JEDEC standard budget is every 64 ms. For an 8,192-row array that means a refresh command on average every 7.8 µs, during which that part of the memory cannot answer you. "Dynamic" means "forgets unless reminded."
SRAM cell
6 transistors · self-holding · ~1 ns read
DRAM cell
1 transistor + 1 capacitor · leaks · refresh every 64 ms
How lopsided is the trade? CS:APP's summary table puts SRAM at 6 transistors per bit against DRAM's 1, roughly 10× slower access for DRAM, and a relative cost gap of about 1,000×per bit in DRAM's favor; Drepper adds that the silicon area of one DRAM cell is "many times smaller" than an SRAM cell's, and that its regular structure packs more densely too. That gap is the reason your L1 cache is measured in kilobytes while your DIMMs are measured in gigabytes. Pick two of fast, big, cheap; you can never have three.
interactive · coming soon
TradeoffTriangle: fast / big / cheap
Planned interactive: a triangle slider where you drag a design point between fast, big, and cheap, and watch what memory technology you just invented. L1 SRAM at one corner, DRAM at another, disk at the third.
SourcesCS:APP 3e §6.1.1 · pp. 617–618Drepper 2007 §2.1.1–2.1.2 · pp. 5–6
The mechanism › Inside the DRAM chip
Rows, sense amplifiers, and why DRAM reads in bulk
The addresses from chapter 0.1 don't map to isolated capacitors. DRAM cells are organized as two-dimensional arrays; modern chips group them into banks, up to 16 in DDR4, each a matrix of rows and columns. An incoming address is split in half: row number first, column number second. The split is not aesthetic. Addressing a gigabit of cells individually would take 30 dedicated address lines and a decoder whose size grows exponentially with them; a square array served by a row decoder and a column selector needs half the address pins, sent in two steps (the classic RAS then CAS strobes). The two-step dance costs access time, a trade of latency for pins that DRAM has made since its earliest days.
Reading is a three-act play. Activate: the row address fires, and an entire row, thousands of cells, dumps its charge onto sense wires that were precharged halfway between 0 and 1; sense amplifiers detect which way each tiny voltage tipped and latch the result. The latched row is the row buffer. Column access: the column address then selects a slice of the buffer to stream out. Precharge: before any other row in the bank can be read, the bank is closed and its sense wires reset.
Two consequences you will meet again and again. First, the read is destructive: sensing drains the capacitors, so the row must be written back from the sense amps; refresh itself is just this activate-and-write-back cycle run on a timer, and controllers budget it to under ~5% of the time. Second, once a row is open, columns within it are cheap, since the data is already sitting in the sense amps, and SDRAM added burst mode precisely so one column command streams out many transfers in a row. Drepper says the quiet part out loud: the burst lengths exist to fill entire cache lines without a new RAS/CAS sequence. On a 64-bit-wide DIMM channel, a burst of 8 transfers is 64 bytes, delivered as a unit. DRAM is physically disinclined to hand you one byte; it deals in cache-line-sized installments whether you wanted them or not.
And the numbers move slowly. A random access that must open a new row on a 2016 DDR4 chip costs about 26 ns of RAS+CAS time, or 39 ns if a precharge is needed first, figures that improved only ~3× across the sixteen years from DDR1, roughly 7% a year. Hold that thought; it becomes chapter 0.4.
Predict first
A DRAM chip stores your bit as:
SourcesCS:APP 3e §6.1.1 · pp. 618–619Drepper 2007 §2.1.3, §2.2.1–2.2.3 · pp. 6–10CAQA 6e §2.2 · pp. 86–89
The number › Your turn
Why your L1 isn't 16 gigabytes
Compute the number › what 16 GB of SRAM would cost
| 16 GB in bits | 16 × 2³⁰ × 8 ≈ 1.4 × 10¹¹ bits |
| SRAM transistors per bit | 6 |
| transistors needed | ≈ 8 × 10¹¹ |
| transistors in a big CPU die | ~10–20 × 10⁹ (order of magnitude) |
| dies of pure SRAM required | 8 × 10¹¹ ÷ ~1.5 × 10¹⁰ |
| Result | ≈ 50 entire CPU dies of nothing but SRAM |
Napkin arithmetic, order-of-magnitude on purpose, but the conclusion is robust, and it agrees with the book numbers: a 1,000× cost-per-bit gap means building main memory out of the fast stuff would take dozens of the largest dies ever manufactured, per machine.The hierarchy isn't a design preference. It's the only geometry the physics allows.
The full story › banks, timing, and what we simplified
Real DRAM is a hierarchy of its own: channels → DIMMs → chips → banks → rows, with each bank owning a private row buffer so the controller can overlap one bank's activation with another's transfer, as many open rows in flight as there are banks. Modern controllers juggle these deliberately, avoiding row reopens and preferring block transfers. That is a preview of the "hide latency with parallelism" move Topic 1 shows GPUs making at massive scale. The full timing alphabet of CL, tRCD, tRP, tRAS, and the w-x-y-z module notation lives in Drepper §2.2 (2007-era bus specifics; the cell physics and protocol structure are unchanged).We deliberately stopped at row + sense amps + burst, because that is exactly the machinery chapter 0.5's cache lines are shaped around. DDR5 reorganizes the channel into two independent subchannels with longer bursts and lands on the same 64-byte unit; the constant survives every generation. And HBM, when you meet it in Topic 1, is not exotic: the same DRAM dies, stacked up to 8 high in-package next to the processor, with transfer rates around 1 TB/s. It is this physics with the bus made very, very wide.
Fast memory and big memory are different circuits: six self-holding transistors versus one leaky ~30 fF capacitor. The gap, ~10× in speed and ~1,000× in cost per bit, is a law of geometry, not a pricing decision, and the memory hierarchy is computing's only possible response to it.