Computer Architecture and DRAM Quiz

Big Data Systems - Modern Hardware I

• Big Data Systems course, Modern Hardware I, taught by Martin Boissier at the Hasso Plattner Institute, University of Potsdam.
• The course covers data processing on modern hardware.
• Topics include hardware-aware data processing, brief introduction to CPU architecture, caches, and data layout.
• Resources for the course include: Data Processing on Modern Hardware - Summer Semester 2022, and Structured Computer Organization by Andrew S. Tanenbaum and Todd Austin.

Timeline II

• The course timeline includes various topics and activities.
• Topics such as ML Systems II, Modern Hardware II, Modern Hardware I, Modern Cloud Warehouses, and an industry talk (by Markus Dreseler, Snowflake) are scheduled.
• The timeline also includes exam preparation and the actual exam.

This Lecture

• This lecture covers hardware-aware data processing.
• It also includes a brief introduction to CPU architecture and discussion of caches and data layout.
• Sources are listed as Data Processing on Modern Hardware - Summer Semester 2022 and Structured Computer Organization, by Andrew S. Tanenbaum, Todd Austin.

Where Are We?

• The current focus is on efficient use of current hardware, trends in hardware, and hardware/software codesign.
• The presentation shows a diagram representing the relationship between application/query language/analytics/visualization, data processing, data management, file system, virtualization/container, OS/scheduling, hardware, Big Data Systems, infrastructure.

Classic DBMS Architecture

• Classic database engines' performance was limited by disk I/O.
• Optimization efforts primarily focused on reducing disk I/O.
• Typical architecture used a buffer pool, row-oriented data structures organized on pages, and tuple-at-a-time processing (Volcano model).
• There's a significant performance gap between registers, caches, main memory, SSD, HDD and archive.

Game Changer - Cheap RAM

• Over time, affordable RAM capacity increased dramatically.
• Modern servers have terabytes of main memory.
• Most database fit comfortably in main memory.
• However, traditional database architectures aren't optimized for in-memory setups.

Multicore CPUs

• High Parallelism using multiple cores/threads, or same instruction on multiple data items (vectorization).
• High Memory Bandwidth- Aggregate memory bandwidth, DDR5 with multiple channels, and Non-Uniform Memory Access (NUMA) architecture.
• Cache coherent memory across all CPUs.

Processor Trends

• Processor trends show a continuous increase in transistor count, performance (number of instructions per second), and number of logical cores.
• However, clock frequencies have historically stagnated and power consumption has increased due to limitations of Dennard scaling (power consumption stays the same while transistor density doubles).

The Limitations of Multi-Core

• Recent processors have limited ability to increase core frequencies without sacrificing power efficiency.
• Increasing core count is a temporary measure to improve performance.
• Unused cores increase the power consumption of the CPU, which is referred to as dark silicon.

Co-Processors [Accelerators]

• Co-processors supplement CPUs to speed up specific operations.
• Examples of co-processors include GPUs, FPGAs, and ASICs.

Graphics Processing Units

• GPUs were originally designed for image rendering.
• GPUs have shifted to general-purpose use.
• Current designs focus on throughput, not latency, allowing for concurrent execution of many thousands of threads for efficient tasks.
• Key characteristics are High Memory bandwidth (1.5 TB/s), asynchronous thread execution and kernel-based workloads.

Field Programmable Gate Arrays

• FPGAs are configurable integrated circuits with logic gates and RAM blocks.
• Their configurations can be modified after manufacturing for particular applications.
• Used for prototyping, networking, and increasingly in databases and data processing.
• FPGAs known for being energy efficient, highly parallel, have a low clock rate, and are hard to program compared to CPUs.

Time for a Rewrite

• To optimize in-memory performance for data processing, re-design systems from scratch to exploit cache and processor efficiency.
• Specific designs are needed for algorithms (parallel joins/aggregations), data structures (column stores/compression), and processing models (vectorization/query compilation).

Scale Out vs. Scale Up Processing

• Scale-out systems distribute data and processing across multiple nodes.
• Scale-up strategies use a small number of powerful nodes, keeping all data in distributed memory.

Industry Trend

• Special hardware (like TPUs) and tightly coupled designs are emerging as crucial approaches to meet increasing compute demand, particularly in cloud-based environments.

High Level Computer Organization

• A hierarchical organization is essential for modern computer architectures.
• Components like CPU, memory, network, PCIe, disk, and FPGA connect via specialized busses and interconnects (ring or mesh).

Hardware Trends

• CPU performance increases much faster than memory performance and leads to significant performance gaps.
• This necessitates special attention to memory performance optimization approaches.

Memory ≠ Memory

• Dynamic RAM (DRAM) stores data as charges on capacitors that need periodic refreshing to prevent data loss;
• Static RAM (SRAM) uses bistable latches for stable storage and doesn't require refreshing, but consumes more energy and chip area.

DRAM Characteristics

• DRAM refresh is important for data retention in DRAM
• The process itself adds a significant time delay to reading data in DRAM from the physical memory
• DRAMs are typically organized as two-dimensional arrays which enables reading several words in parallel, which is an important concept for reading data in faster time.

SRAM Characteristics

• SRAM access is very fast because of the nature of its cell state storage
• This high speed comes at the cost of higher cost and higher area footprint compared to DRAM.
• Therefore, SRAM is commonly used as cache for faster storage.

Memory Hierarchy - Large Scale

• A hierarchy of memory types (registers, caches, main memory, SSD, HDD, archive) with varying access speeds and capacities.

Caches

• Caches speed up access to frequently accessed data, improving performance.
• Spatial locality and temporal locality are important principles for caching.

Principle of Locality

• Data accessed frequently tends to be located near each other (spatial locality)
• Program logic often revisits the same code area over and over (temporal locality).

Memory Access Example

• Accessing data arranged row-wise vs. column-wise can impact cache performance and memory access time.

Memory Access

• CPU checks cache for data; if found (cache hit), data is accessed quickly.
• If not found (cache miss), the CPU reads from a lower memory layer.
• CPU stalls until data becomes available.

Cache Performance

• The cost of a cache hit is much lower than a cache miss, potentially by hundreds of times.

Cache Internals

• Caches are organized in cache lines;
• Only complete lines can be loaded into or evicted from a cache.
• Typical cache line size is 64 bytes.

Cache Organization - Example

• Cache levels (L1, L2) have specific sizes, line sizes, and latencies.

Caches Latencies

• Latency (in cycles) is an important timing concept in memory hierarchical systems, varying across different memory types (register, L1, etc).

Caches on a Die – Intel i7

• Multicore processors have on-die caches for faster data access.

Caches on a Die II – M1

• The M1 processor shows specific cache sizes and configurations designed for its specific architecture.

Numbers on M1

• Benchmarks demonstrate the performance of memory access in the M1 processor, focusing on data size and runtime per element.

DELab Measurements

• Measurements from experiments show runtime differences in various processor architectures when processing data of increasing sizes.

Performance (SPECint 2000)

• Performance metrics demonstrate how different programs access data elements through cache (miss rates), illustrating how the cache behaves under different workloads.

Assessment

• Database systems show poor cache behavior because of poor code locality in polymorphic functions, which is associated with tasks such as resolving attribute types, and because of poor data locality in the manner that database systems visit data.

Cache Friendly Code

• Optimizing code for cache performance is crucial in databases.
• Organize data structures and access patterns to favor cache locality (spatial/temporal.)
• Performance should be benchmarked and optimized for the desired environment.

Data Layout

• Row-based and column-based are two different data storage approaches in databases to optimize cache performance.

Caches for Data Processing

• Data storage models affect cache usage.
• Optimizing for the temporal and spatial locality of data is important.

Data Storage Layout

• Row-oriented and column-oriented data storage models are two main approaches used for storing data in the databases and they have different impact on cache usage.

Full Table Scan

• Row-oriented storage layouts may load irrelevant data into cache during a full table scan, leading to inefficient use of caches.
• Column-based layouts can reduce cache misses related to irrelevant data because they only fetch data blocks/columns required for the query, improving data locality.

Column-Store Trade-off

• Tuple recombination may cause significant overhead within column layouts;
• Hybrid approaches attempt to combine strengths of row- and column-oriented approaches.

Parallelism

• Pipelining enables parallel execution of tasks.
• Separating chip regions lets instructions independently function.
• VLSI limits the ability to increase pipeline parallelism.

Other Hardware Parallelism

• Identical hardware can be controlled by different instructions;
• CPUs use Multiple Instructions, Multiple Data (MIMD).

Vector Units (SIMD)

• Modern processors use SIMD (Single Instruction, Multiple Data)
• SIMD instructions enable executing multiple data operations simultaneously.

Vectorized Execution in a Nutshell

• Scalar instructions operate on data one at a time;
• Vector instructions operate on several data items simultaneously.

SIMD Instructions

• SIMD instructions process multiple data items with a single instruction, leveraging vector capabilities in modern CPUs.

Example: SIMD vs. SISD

• Examples illustrate how SIMD processes multiple data elements simultaneously, whereas, in contrast, SISD operates on data one at a time.

SIMD in DBMS Operations

• Vectorized database operations (like table scans, joins and sorts) can leverage SIMD for accelerated execution.

Hyrise - In-Memory Database System

• Open-source project aimed at in-memory database system optimization.

Summary

• Course summary emphasizing hardware-aware data processing, and relevant topics in caching/CPU architecture and memory access.

Hardware-Conscious Data Processing

• Overview of a course or module focusing on optimizing data processing and its performance for modern computer architectures.
• This includes basic database concepts, performance analysis, and optimizing memory-related aspects.

Next Part

• Continuing with modern hardware, and showing a hierarchy diagram for the application and the data needed by the applications to function.

Questions?

• Various methods for getting answers to questions about the course or material.

Test your knowledge on computer architecture, including SIMD instructions and DRAM characteristics. This quiz covers various aspects of memory types, cache usage strategies, and instruction sets. Perfect for students studying computer science or electrical engineering.