Computer Architecture: Memory Bus Performance

Memory Systems

• The cost of quadrupling the memory bus may become prohibitive and would not yield that much performance improvement.
• A 256-bit bus without interleaving would give an effective cycle of 2.92, resulting in a speedup of just 1.06 with respect to the memory interleaving configuration of 4 banks.

RAM Construction Technology

• ROM is a non-volatile memory that can be written just once, with some variations being electronically erased and reprogrammed at slow speed (EEPROM).
• SRAM prioritizes speed and capacity, with data not needing to be refreshed periodically, and is about 8 to 16 times more expensive than DRAM.
• DRAM prioritizes cost per bit and capacity, with data needing to be refreshed periodically, and has multiplexed address lines.

Memory Modules

• Memory modules are used in the form factor of DIMM with 4 to 16 memory chips in a 64-bit bus to facilitate handling and exploit memory interleaving.
• The number of pins varies among different DDRn.

Virtual Memory

• Virtual memory automatically manages the two levels in the memory hierarchy represented by the main memory and the secondary storage (disk or flash).
• Virtual memory provides memory space sharing and protection, and memory relocation.
• The focus on virtual memory is specific to the computer architecture point of view, with concepts analogous to caches.

DDR SDRAM

• DDR SDRAM was an innovation where memory data is transferred on both rising and falling edges of the SDRAM clock signal, thereby increasing the data transfer rate.
• DDR technology has evolved with increased clock rates and voltage reduction in chips, from DDR1 to DDR5.

DRAM Organization

• Modern DRAM is organized in banks with up to 16 banks in the DDR4, with each bank having a number of rows.

DRAM Operation

• The DRAM controller manages address multiplexing by sending bank and row numbers followed by the column address, and finally, the data is accessed for a reading or a writing process.
• The activate command (ACT) opens a bank and a row, and loads the entire row into the row buffer.
• The precharge command (PRE) closes the bank and row and prepares it for a new access.

DRAM Performance

• DRAM performance is affected by non-uniform access time due to data location and the refresh/write back requirement.
• The time is divided into: (i) RAS precharge, (ii) RAS-to-CAS delay, (iii) CAS latency, and (iv) cycle time.

Cache Memory Organization

• The tag field in a cache block includes an additional bit to indicate the block's data validity, also known as the valid bit.
• Tags are searched in parallel for speed purposes.

Block Replacement

• In the case of a cache miss, the cache controller selects a block to be replaced with the required new data.
• There are three main replacement strategies: random, least recently used (LRU), and first-in, first-out (FIFO).
• Random replacement aims to spread allocation uniformly by randomly selecting candidate blocks.
• LRU replacement keeps track of block accesses to reduce the chance of throwing out data likely to be needed soon.
• FIFO replacement determines the oldest block rather than the least recently used block.

Block Placement

• Cache organizations can be fully associative, direct mapped, or set associative.
• In fully associative caches, a block can be placed anywhere in the cache.
• In direct mapped caches, each block has only one place it can appear in the cache, determined by Equation (4.1).
• In set associative caches, a block can be placed in a restricted set in the cache, determined by Equation (4.2).
• A block is first mapped onto a set and then placed anywhere within that set.

Cache Organization Comparison

• Table 4.1 shows pros and cons of different cache organization aspects, including fully associative, direct mapped, and set associative caches.
• Fully associative caches offer full flexibility but have high complexity and implementation cost.
• Direct mapped caches are simple but may have possible inefficiency due to inflexibility.
• Set associative caches are a compromise between flexibility and complexity.

Write Strategy

• Writes are approximately 10% of the memory traffic.
• Reads dominate processor cache accesses, and enhancing read performance is crucial.
• Blocks can be read from the cache at the same time the tag is read and compared.
• In case of a hit, the requested part of the block is given to the processor immediately.
• The write strategy cannot be applied in the same way as the read strategy since the tag must be checked to confirm whether the address is a hit.

Calculate the effective cycle time for a 256-bit memory bus without interleaving, and discuss the cost and performance implications of increasing memory bus width.