What a memory is

A memory is two things together: a collection of cells that store binary information, and the circuits that move information into and out of those cells. A cell holds one bit. Cells are grouped into rows called data words, and each word has an address.

Three buses connect the processor to memory.

• The address bus selects the cell or word. Its width fixes how many locations can be named.

• The control bus carries the operation, principally read or write, plus strobes such as RAS and CAS on a DRAM.

• The data bus carries the word. On a write the processor drives it. On a read the memory drives it and the processor senses it.

A read or write is always the same three steps.

1. The processor places the address on the address bus. Decoding circuitry selects the device and the location inside it.

2. The processor places a read or write indication on the control bus.

3. The data word moves across the data bus, from memory to processor or from processor to memory.

This is the von Neumann arrangement: one memory holds both instructions and data, and the processor fetches from that memory over the buses. The practical consequence is the memory wall. Processors issue requests much faster than a large DRAM array can answer them, so the rest of this subject is a set of techniques that hide latency and raise throughput: caches, synchronous interfaces, bursts, double data rate, bank interleaving, and extra channels.

Primary and secondary memory

Primary memory sits on the processor’s buses and holds the code and data of the running machine. RAM and ROM are primary. Access is by an address, and the access time is essentially independent of which address was used last.

Secondary memory is the large permanent store: magnetic tape, hard disk, optical disc, and, in current machines, flash SSDs used as disks. It is bigger and it keeps its contents with the power off. It is not wired as a random-access array on the processor bus. The processor reaches it through a controller, usually a block at a time.

The course history is short and worth remembering as a change of technology, not a change of the bus idea.

• From the 1950s to the 1970s the working store was magnetic core. Each bit was a tiny ferrite ring.

• In 1970 Fairchild shipped a relatively large semiconductor memory: a 256-bit SRAM, about the size of a core plane and much faster.

• Also in 1970 Intel shipped the 1103, the first commercial DRAM, organised as 1024 bits and smaller still.

• By 1974 a semiconductor bit cost less than a core bit, and core disappeared from new designs.

How data are reached

“Random access” is an access method, not a brand of chip. Four methods appear in the notes.

Access methods

Method	Rule	Time depends on	Example
Sequential	The medium is read in the order it was written, from the start or from the current position.	Where the item sits, and where the head already is.	Tape
Random	Every location has a wired address. Any location is selected directly.	Nothing about previous accesses. The time is constant.	Main memory
Direct	A block is selected at random; inside the block the scan is sequential.	The block seek, then the offset inside the block.	Disk
Associative	The item is named by a key, not by an address. The memory itself turns the key into a location.	Neither location nor history. Retrieval time is constant.	CAM, cache tags

A disk is the clearest direct-access case. The controller can jump to a chosen sector, which is the random part. Bytes inside that sector then come off the platter in order, which is the sequential part. A cache lookup is the associative case used later: the processor presents an address, and comparators ask “does any stored tag equal this key?” rather than indexing a unique cell by that whole address.

Semiconductor memory families

Read-only memory

ROM keeps its contents when power is removed. It is used for code and constants that must be present at power-up: boot firmware, lookup tables, and device identity. The name “read-only” describes the system’s normal use. Several of the technologies below can be rewritten, just not at RAM speeds and not for an unlimited number of cycles.

Random-access memory

RAM here means a memory that can be read or written at any address, at any time, for as long as the power is on. The slides list five kinds.

Static RAM

An SRAM cell is a bistable latch: two inverters connected in a loop, so each holds the other in a stable state. The smallest latch uses four transistors. A practical cell adds two access transistors that connect the latch to a differential bit-line pair, giving the standard 6T cell. Some dense variants use four transistors plus resistive loads. Either way, several transistors stand at every bit.

Consequences, which are the whole reason SRAM and DRAM coexist:

• The state is static. Nothing leaks away, so the cell needs no refresh, and it can be read at any moment.

• The cell is large, draws static current in older processes, and costs far more per bit than DRAM. The 2012 course figures are about 0.5–2.5 ns access and roughly $500–$1000 per gigabyte, against DRAM at 50–70 ns and $10–$20 per gigabyte. Absolute prices have fallen since 2012; the ratio has not disappeared.

• SRAM is the technology of caches, register files, and small on-chip buffers, where a few megabytes of very fast memory are worth the area.

Dynamic RAM

A DRAM cell stores the bit as charge on a capacitor, and it uses one transistor as the switch. This is the 1T1C cell.

Write. The sense amplifier, or the write driver, drives the bit line to the required level (high for 1, low for 0). The word line turns the transistor on. Current flows until the capacitor sits at that voltage. The word line then turns off and the charge is trapped, apart from leakage.

Read.

1. The bit line is precharged to a midpoint voltage and left floating.

2. The word line turns the transistor on. The tiny cell capacitance shares charge with the much larger bit-line capacitance, so the bit line moves only by a few tens of millivolts.

3. A sense amplifier compares that movement with a reference and decides 0 or 1.

4. The decision is driven back onto the bit line, which restores the capacitor. Without this write-back the read would destroy the bit.

The charge leaks through the transistor and the capacitor dielectric even while power is applied. That leak is what the word dynamic refers to. Every row must be read and restored periodically. This is refresh. While a bank is refreshing, the processor cannot use that bank. The original IBM PC used the Intel 8253 timer to start DRAM refresh cycles. A modern controller hides the same job: a typical commercial DRAM refreshes each row within a 64 ms window, inserting a refresh command about every 7.8 μs.

The other RAM names on the slide

Organisation, capacity, and multiplexing

Organisation is not capacity

Cells are arranged as a two-dimensional array. One row is a data word. The row’s identity is its address.

Organisation is the shape of the rectangle; capacity is the area. Knowing that a chip holds 8 Mbit does not say whether it is \(2^{20}\times 8\) or \(2^{23}\times 1\). The shape decides the pinout and how the chip is wired onto a module.

Byte-addressable machines, which means essentially every machine in this course, give each address one byte. The logical picture of main memory is therefore an array of 8-bit items. A wider physical chip, or several chips side by side, still presents that byte array to the programmer. The hardware extracts the requested byte from a wider transfer.

Why DRAM multiplexes the address

A large DRAM has a wide internal address. Bringing every address bit out to its own pin makes the package large, and the package, not the silicon, then limits how many chips fit on a module. The fix is to reuse the pins.

The internal address is split into a row and a column, each of \(x/2\) bits.

1. The row is placed on the address pins and latched by the row-address strobe, RAS.

2. The column is placed on the same pins and latched by the column-address strobe, CAS.

The pin count for the address falls from \(x\) to \(x/2\), plus the two strobes. A multiplexer outside the chip selects which half is driving the pins. A demultiplexer inside the chip steers the latched row to the word-line decoder and the latched column to the bit-line selection.

The array really is two-dimensional. Raising RAS selects one row and copies that entire row into the sense amplifiers. Raising CAS then picks one column out of the open row and connects it to the data pins. This is why a later column read to the same row can skip the row step, and why closing the row (precharge) is a separate event.

Synchronous DRAM

What the clock changes

Asynchronous DRAM of the 1980s and early 1990s ran as a chain of delays. The processor presented a row, waited a datasheet minimum, presented a column, and waited another minimum. Nothing on the chip was aligned to a shared clock.

Two intermediate steps are named on the packaging slides and are easy to lose:

• Fast page mode (FPM). After RAS has opened a row, several column addresses can be issued with CAS alone. The row address is not repeated. This is the hardware form of “the page stays open”.

• Extended data out (EDO). The output buffer holds the previous column’s data valid while the next column address is set up, so the column cycles overlap. EDO is a small bandwidth gain over FPM on the same array.

SDRAM puts a clock between the processor and the DRAM. Address, data, and commands are sampled on clock edges, and every delay in the timing diagram is an integer number of those clocks. The array itself is still the same capacitor cell. Synchrony is an interface change. It is what makes bursts, predictable latency, and later double-data-rate transfers possible.

The four timing parameters

A timing set is written \[t_{\mathrm{CL}}\text{-}t_{\mathrm{RCD}}\text{-}t_{\mathrm{RP}}\text{-}t_{\mathrm{RAS}}\] and every term is a count of I/O bus clocks. If the I/O frequency is \(f_{\mathrm{I/O}}\), \[t_{\mathrm{CK}}=\frac{1}{f_{\mathrm{I/O}}},\] and a parameter of \(n\) clocks lasts \(n\cdot t_{\mathrm{CK}}\) seconds. The course example is a DDR2-800 part at 5-5-5-18: the I/O clock is 400 MHz, \(t_{\mathrm{CK}}=\SI{2.5}{ns}\), so \(t_{\mathrm{CL}}=t_{\mathrm{RCD}}=t_{\mathrm{RP}}=\SI{12.5}{ns}\) and \(t_{\mathrm{RAS}}=\SI{45}{ns}\). Those are guaranteed maxima. The silicon may be faster; the controller must not assume that it is.

Command spacing on an open and closed row

Symbol	What must be true	From \(\to\) to
\(t_{\mathrm{RCD}}\)	After Activate, the sense amplifiers need this long before a column command.	Activate \(\to\) Read or Write
\(t_{\mathrm{CL}}\)	CAS latency. Clocks from the Read command to the first data beat.	Read \(\to\) first data
\(t_{\mathrm{RP}}\)	RAS precharge. After Precharge, the bit lines must recover before that bank can be activated again.	Precharge \(\to\) next Activate
\(t_{\mathrm{RAS}}\)	Active-to-precharge delay. The row has to stay open at least this long. A Precharge issued earlier is illegal.	Activate \(\to\) Precharge

Two derived intervals are used constantly in datasheets, even though they are not part of the four-number set.

• Row cycle time \(t_{\mathrm{RC}}=t_{\mathrm{RAS}}+t_{\mathrm{RP}}\). This is the minimum time from one Activate of a bank to the next Activate of the same bank, when the row is closed in between.

• If the access hits a row that is already open, \(t_{\mathrm{RCD}}\) has already been paid. Only \(t_{\mathrm{CL}}\) stands between the Read and the first data. That is a page hit. A page miss pays \(t_{\mathrm{RP}}+t_{\mathrm{RCD}}+t_{\mathrm{CL}}\) because the old row is closed and the new row is opened.

A closed-page read, then a burst

From a long idle (every bank precharged) the controller does the following.

1. Issue Activate with the row address. Wait \(t_{\mathrm{RCD}}\).

2. Issue Read with the column address. Wait \(t_{\mathrm{CL}}\).

3. Data words then leave on successive beats. In a burst, only the first column address is supplied. The chip’s internal counter walks the next columns. The processor does not pay address setup time on each beat. The number of beats is the burst length.

4. If the page must be closed, issue Precharge, but not before \(t_{\mathrm{RAS}}\) has elapsed from the Activate, and then wait \(t_{\mathrm{RP}}\) before that bank is free.

On single-data-rate SDRAM, one word leaves on each clock. On DDR, two words leave on each I/O clock, one on the rising edge and one on the falling edge. For a burst of \(N\) words on DDR, the first word is valid at \[t_{0}=(t_{\mathrm{RCD}}+t_{\mathrm{CL}})\,t_{\mathrm{CK}}\] measured from Activate, and word \(k\) (from 0) is valid at \[t_{k}=t_{0}+k\cdot\frac{t_{\mathrm{CK}}}{2}.\] The last word is \(k=N-1\), so the transfer completes at \[t_{\mathrm{last}}=\left(t_{\mathrm{RCD}}+t_{\mathrm{CL}}+\frac{N-1}{2}\right)t_{\mathrm{CK}}.\] The burst occupies \(N/2\) clocks, and the last beat sits on the falling edge, half a clock before a naive “add \(N/2\)” would place the end.

Interleaving hides precharge

A bank that has just been precharged cannot answer a new Activate until \(t_{\mathrm{RP}}\) is over. If consecutive words live in different banks, the controller activates the next bank while the previous bank is still finishing, and the precharge of one bank sits underneath the access of another.

The course picture is four banks producing four consecutive words \([i]\), \([i+1]\), \([i+2]\), \([i+3]\) on successive beats, each bank having been opened on its own. The two-bank teaching case is the same idea with bytes: even addresses in bank 0, odd addresses in bank 1, so instruction fetch, which walks addresses in order, alternates banks and never waits out a precharge in the steady state.

Burst mode and interleaving solve different problems. A burst removes repeated column addresses inside one open row. Interleaving removes the precharge gap between rows, or between banks, by having another bank ready.

Double data rate and the prefetch

DDR SDRAM keeps the same command protocol and sends data on both clock edges. That doubles the number of transfers per I/O clock without doubling the clock frequency the array has to track.

Later generations do not make the capacitor array twice as fast. They widen the internal fetch and serialise it onto a faster I/O:

• DDR prefetches 2 bits per pin per memory clock, and the I/O clock equals the memory clock. Two beats per I/O clock give two transfers per memory clock.

• DDR2 prefetches 4. The I/O clock is twice the memory clock, and each I/O clock still carries two beats, so four transfers leave per memory clock. The course slide’s phrase is “4 data in 2 cycles”. Core voltage falls from 2.5 V (DDR) to 1.8 V. Typical CAS latency rises to about 4–6 cycles, from about 2–3 on DDR. The extra cycles are the price of the deeper prefetch and the higher I/O rate.

• DDR3 prefetches 8. The I/O clock is four times the memory clock. Eight transfers leave per memory clock. Voltage is 1.5 V (1.35 V on DDR3L).

• DDR4 keeps the 8-bit prefetch and raises the I/O rate further, at 1.2 V, with bank groups so that back-to-back column commands can go to different groups.

• DDR5 splits each module into two 32-bit sub-channels, lowers the rail to about 1.1 V, and adds on-die ECC. It is the current desktop generation; the course slides stop their worked table at DDR3.

Power is the reason the voltage keeps falling. Dynamic power scales as \[P \propto C V^{2} f.\] A lower \(V\) buys a higher \(f\) inside the same thermal budget. The \(V^{2}\) term is why a drop from 1.5 V to 1.2 V is worth more than the \(20\%\) the voltages themselves suggest.

The course table, with the arithmetic made explicit

Standard name	I/O clock	Memory clock	Transfers	Data rate	Module name
	(MHz)	(MHz)	(MT/s)	(MB/s)
DDR-400	200	200	400	3200	PC-3200
DDR2-1000	500	250	1000	8000	PC2-8000
DDR2-1066	533	266	1066	8533	PC2-8500
DDR3-1866	933	233	1866	14933	PC3-14900
DDR3-2133	1066	266	2133	17066	PC3-17000

The PC number is sometimes rounded to a marketing value. DDR2-1066 really moves \(1066\times 8=8528\) MB/s and is sold as PC2-8500; DDR3-1866 is sold as PC3-14900 rather than 14928. When the division \(Z/8\) is not an integer that the standard recognises, use the advertised standard name that the manufacturer pairs with that module, and show the rounding.

Where the speed actually comes from

The course summary of high-performance DRAM is four techniques, and they should not be collapsed into one “faster chip” claim.

• Interleaving reduces observed latency by hiding precharge behind another bank’s access.

• Burst mode raises throughput on sequential addresses by deleting per-word address overhead.

• Double data rate, and the deeper prefetch behind DDR2 and DDR3, raises the number of bits per pin per second.

• Multi-channel memory adds whole buses. Two independent channels, populated and running, double the peak bandwidth of one channel. High-end chipsets add a third or a fourth channel.

Packaging and the module hierarchy

From DIP to DIMM

JEDEC fixes the card length, the pin count, and the position of the keying notch. The notch is why a DDR3 module will not seat in a DDR4 socket. Forcing it damages both.

Buffered and unbuffered modules

The slides mark this section as not required for memorisation. The mechanism is still short enough to keep, because it explains the cost order.

The cost order on the slide is LRDIMM, then RDIMM, then UDIMM, because each step up adds onboard buffering that the cheaper part does not have.

Channel, DIMM, rank, chip, bank

The slide gives a strict containment, from the widest structure down to the row: \[\text{channel} > \text{DIMM} > \text{rank} > \text{chip} > \text{bank} > \text{row and column}.\]

A request address, once the memory controller has decoded it, therefore names a channel, a rank on that channel, a bank inside the chips of that rank, a row, and a column. The programmer’s byte address is mapped onto that hierarchy by the controller. The programmer does not issue RAS.

HBM, embedded DRAM, and GDDR

These three are DRAM cells with a different package or a different I/O, aimed at bandwidth rather than at a socket on a motherboard.

High bandwidth memory

HBM stacks several DRAM dies and places the stack in the same package as the processor, either beside it on a silicon interposer or, in later packaging, closer still. Thousands of wires, much shorter than a DIMM trace, run between the processor and the stack. Each wire is not especially fast. The width is the point: a single HBM stack is on the order of a 1024-bit interface, against 64 bits for a DIMM.

The course slide dates the first widespread announcements from AMD, Intel, and Nvidia to mid-2017 and points at Figure 2.7 of Hennessy and Patterson, sixth edition. The slide’s caption uses the word eDRAM for this stacked or adjacent package. The next slide uses eDRAM for a different construction. Keeping the two apart matches the datasheets:

• HBM is a stack of ordinary-style DRAM dies, built on a DRAM process, sitting next to the compute die and connected by an interposer or by hybrid bonding.

• Embedded DRAM is DRAM fabricated in the logic process, on the same die as the processor.

Embedded DRAM

eDRAM is on the CPU die, so the wire delay and the pin limit disappear. The bandwidth is high enough to back an on-die GPU. Intel used it from Haswell onward; the course example is a Coffee Lake part with 128 MB of eDRAM for the integrated GPU.

The costs are the ones the slide lists. The extra capacitance and the refresh current sit on the same die as the cores, so the thermal design has to absorb them. A DRAM process is denser for capacitors than a logic process, so an eDRAM megabyte costs more area, and more money, than a megabyte on a commodity DDR die. eDRAM is a bandwidth tool for a hot spot, not a replacement for the DIMM slots.

GDDR

GDDR is a DRAM family tuned for a GPU’s frame buffer.

• The interface is wider than a single desktop DRAM chip, so more bits move per clock.

• The data pins are clocked faster than a DIMM channel of the same era. The parts are soldered next to the GPU. A socket and a long trace would not hold the signal.

• The I/O voltage is lower, which is required once \(P\propto CV^{2}f\) is taken seriously at those pin rates.

The generation numbers are not the desktop DDR numbers. The course mapping is:

A GDDR5 device is not “DDR5 for graphics”, and a GDDR6 device is not DDR6. DDR5 is a later, separate JEDEC standard for system memory.

Low-power DDR

LPDDR is the DRAM used in phones, tablets, and a growing set of laptops and cars. The cell is still a capacitor. The differences are a lower voltage, a narrower channel that can be turned off in pieces, and extra power states for the long intervals when a phone is doing nothing. This chapter follows the SK hynix video LPDDR, Evolving to Your Needs, which is the source named in Assignment 1.

LPDDR4(x) against LPDDR3

The video organises the step from LPDDR3 to LPDDR4(x) as three changes.

1. Two short \(\times 16\) channels replace one long \(\times 32\) channel. An LPDDR3 die is a single 32-bit channel. The clock has to cross the whole data path, and that length costs core power on every read and write. LPDDR4(x) divides the die into two 16-bit channels, each with its own command/address bus. Each clock tree is short. Core power drops, and the pin rate can rise. The video’s headline rate for LPDDR4(x) is 4266 Mbps per pin.

2. The I/O becomes LVSTL at 0.6 V. LPDDR3 drives the pins with HSUL12: a 1.2 V swing and optional on-die termination. LPDDR4(x) uses low-voltage swing terminated logic. \(V_{\mathrm{DDQ}}\) is 0.6 V, termination is sized for that swing, and the I/O energy per bit falls. A \(1\mathrm{ynm}\) part in the video is quoted at up to about \(10\%\) less power than the previous generation at the same 4266 Mbps headline.

3. A failed row can be repaired after packaging. Post-package repair replaces one bad row per bank, under a vendor guard key in a mode register, so a customer command cannot fall into the repair state by accident. The repair is a row operation: enter the mode, activate the failed address, run the repair, precharge, and leave. Mode-register state from before the repair is discarded on the way out. The point of doing this after packaging is that some defects only appear once the die is in its package.

LPDDR5 against LPDDR4(x)

The video’s LPDDR5 section is again three mechanisms, after a comparison table. The headline numbers on that table are a per-pin rate of 400–6400 Mbps, a single \(\times 16\) channel, a DDR command bus, and link ECC on top of on-die ECC.

1. WCK, a data clock that is not the command clock. LPDDR4 times everything from CK. LPDDR5 sends commands on CK and sends data against a separate differential clock, WCK, running at two or four times CK. CK is held to 800 MHz or below even when the pins run at 6.4 Gbps. The WCK tree is short, so the jitter at the data pins stays small enough for that rate. The command bus, now DDR and only seven bits wide, toggles far less often, which cuts active standby power. 6400 Mbps is the \(50\%\) gain over 4266 Mbps quoted in the video.

2. The bank layout is programmable. The mode register selects one of three organisations:

• bank-group mode, four groups of four banks, for rates above 3200 Mbps, with burst length 16 or 32 and a 2 KB page;

• 16-bank mode, no groups, up to 3200 Mbps;

• 8-bank mode, burst length 32 only, a 4 KB page, any speed.

The controller picks the layout that matches the traffic. Sequential traffic likes a long burst and a wide page. Random traffic likes more banks, so that a miss in one bank overlaps a hit in another.

3. DVFS drops the supplies when the link is not at full speed. Most accesses are not at the peak rate. DVFS-C moves peripheral circuitry onto a 0.9 V rail (\(V_{\mathrm{DD2L}}\)) at low speed, instead of the 1.05 V core rail. DVFS-Q drops \(V_{\mathrm{DDQ}}\) from 0.5 V to 0.3 V in the same region. When the link returns to high speed, both rails return to the fast values. The energy saving is concentrated where the phone actually spends its time.

Figures read from the video’s comparison table

	LPDDR4(x)	LPDDR5
Data rate per pin	400–4266 Mbps	400–6400 Mbps
Clocking	CK only	CK for commands, WCK for data
Command bus	6-bit SDR, with CKE	7-bit DDR, no CKE
\(V_{\mathrm{DDQ}}\)	1.1 V or 0.6 V	0.5 V or 0.3 V
I/O organisation	two \(\times 16\) channels	one \(\times 16\) channel
Banks	8 per channel	8-bank, 16-bank, or bank-group
Prefetch	16	16, or 32 in 8-bank mode
ECC	on-die	on-die and link

Set-associative caches

The DRAM chapters explain the main store. The processor rarely talks to it directly. A cache is a small, fast SRAM that keeps copies of recently used blocks. This chapter is the minimum needed to turn a partial cache description into a full one, which is the skill Assignment 1 tests. The placement rule throughout is set-associative.

The three fields of an address

Main memory is byte-addressable. The cache does not store individual bytes in isolation. It stores a block (also called a line): a contiguous run of bytes that travel together on a miss.

A set-associative cache divides its storage into sets. Each set holds a fixed number of blocks, called the ways. The degree of associativity is the number of ways. The depth of the array is the number of sets. That is the reading used here: the index selects a row, and the ways are the columns of that row.

An address splits into three fields, from the most significant bit downward. \[\begin{array}{c|c|c} \text{tag} & \text{index} & \text{offset} \\ \end{array}\]

• The offset selects the byte inside the block. If the block is \(B\) bytes, the offset has \(\log_{2} B\) bits. A 4-byte block has a 2-bit offset. An 8-byte block has a 3-bit offset.

• The index selects the set. If there are \(S\) sets, the index has \(\log_{2} S\) bits. With depth defined as the number of sets, \(\text{depth}=S=2^{(\text{index bits})}\).

• The tag is everything that remains. Two different blocks can map to the same set; the tag tells them apart. \(\text{tag bits}=\text{address bits}-\text{index bits}-\text{offset bits}\).

On a request the hardware does four things.

1. It uses the index to read exactly one set.

2. It compares the incoming tag with the tag stored in every way of that set. The number of comparisons equals the associativity. A direct-mapped cache is the special case of one way, and it does one comparison. A fully associative cache is the special case of one set, and it compares against every block.

3. A match with the valid bit set is a hit. The offset then picks the byte out of the matching block.

4. No match is a miss. The block is fetched from the next level, stored in one way of that same set, and the tag is updated.

The capacity relation

Let \(C\) be the data capacity in bytes, \(B\) the block size in bytes, \(N\) the associativity, and \(S\) the number of sets. Data capacity counts the stored bytes, not the tag or valid bits. \[\begin{align*} \text{number of blocks} &= S \times N,\\ C &= S \times N \times B,\\ S &= \frac{C}{N \times B}. \end{align*}\] Any one of \(C\), \(B\), \(N\), and \(S\) can be recovered from the other three. The address-field widths then follow from \(B\), from \(S\), and from the address width.

What associativity is for

A direct-mapped cache (\(N=1\)) needs one comparator and has a simple timing path. Any two blocks whose index fields match compete for that single slot. That collision is a conflict miss.

Raising \(N\) gives those blocks several slots in the set, so a conflict miss becomes rarer. The cost is \(N\) comparators, a wider tag store, and a multiplexer that picks the matching way. Going from 1 way to 2, and from 2 to 4, removes most of the conflict misses that are worth removing. Going far past that adds comparators faster than it adds hits. A fully associative cache is the limit: excellent placement freedom, and a comparison against every block on every access. Set-associative caches exist because they stop in between.

The tag grows as associativity grows, for a fixed capacity and block size. More ways means fewer sets, so the index shrinks, and the bits that left the index join the tag. Those extra tag bits are exactly the information the comparators need in order to tell the ways apart.

Formula sheet

Quantity	Relation
Chip organisation	\(2^{x}\times y\), with \(2^{x}\) locations of \(y\) bits
Chip capacity	\(2^{x}\times y\) bits. Divide by 8 for bytes.
Multiplexed address pins	\(x/2\), plus RAS and CAS
Module name	\(\mathrm{DDR}X\text{-}Y=\mathrm{PC}X\text{-}Z\), \(Z=Y\times 8\) for a 64-bit DIMM
Data rate \(Z\)	megabytes per second
Transfer rate \(Y\)	megatransfers per second
I/O clock	\(f_{\mathrm{I/O}}=Y/2\)
Memory clock	\(f_{\mathrm{mem}}=Y/\text{prefetch}\), prefetch \(=2,4,8\) for DDR, DDR2, DDR3
I/O period	\(t_{\mathrm{CK}}=1/f_{\mathrm{I/O}}\)
Parameter in seconds	\(n\) clocks \(\times t_{\mathrm{CK}}\)
First data, from idle	\((t_{\mathrm{RCD}}+t_{\mathrm{CL}})\,t_{\mathrm{CK}}\)
Last of \(N\) DDR beats	\(\bigl(t_{\mathrm{RCD}}+t_{\mathrm{CL}}+(N-1)/2\bigr)\,t_{\mathrm{CK}}\)
Do not add on an open page	\(t_{\mathrm{RP}}\), and \(t_{\mathrm{RAS}}\) unless a precharge is being scheduled
Cache offset	\(\log_{2}(\text{block size})\)
Cache index	\(\log_{2}(\text{number of sets})\)
Cache depth	number of sets
Cache tag	address width \(-\) index \(-\) offset
Cache capacity	sets \(\times\) ways \(\times\) block size
Tag comparisons	the associativity
Dynamic power	\(P \propto C V^{2} f\)

Sources

1. ELEC 3630SEF, 01 Memory Basics and Main Memory, Eddie K. C. Tjon, 2026/27. The outline, the organisation formula, the RAS/CAS pin argument, the four timing parameters, the JEDEC naming table, the module hierarchy, and the GDDR generation map are taken from these slides and then written out in full.

2. William Stallings, Computer Organization and Architecture. Access methods, cache field splitting, and the set-associative placement rule.

3. John L. Hennessy and David A. Patterson, Computer Architecture: A Quantitative Approach, 6th ed., 2019, Figure 2.7. The HBM packaging figure cited on the course slide.

4. JEDEC JESD79-2 (DDR2 SDRAM) and the later DDR3, DDR4, DDR5, and LPDDR standard families. Prefetch widths, burst timing, refresh windows, and the LPDDR voltage and bank figures.

5. SK hynix, LPDDR, Evolving to Your Needs, https://www.youtube.com/watch?v=bVOzGVABUZQ. The three LPDDR4(x) changes and the three LPDDR5 changes in Chapter 7, including the comparison-table numbers.

6. Bruce Jacob, Spencer Ng, and David Wang, Memory Systems: Cache, DRAM, Disk. The row, bank, rank, and channel hierarchy, and the reason a DRAM read restores the cell.

Prices per gigabyte and the access times quoted for SRAM and DRAM are the 2012 figures printed on the course slide. They are there to fix the ratio, not as a current market quote.