Computer Architecture and GPU Advances

HPC Introduction

• High-performance computing (HPC) is the ability to perform complex calculations at high speeds, processing data efficiently.
• A supercomputer is a prominent example of an HPC solution.
• Supercomputers comprise thousands of computing nodes that collaborate to perform tasks.
• The IBM Blue Gene/P supercomputer "Intrepid" at Argonne National Laboratory uses ~164,000 processor cores.

HPC Contents

• Introduction to HPC
• Parallel Architectures
• Multi-Cores
• Graphical Processing Units (GPUs)
• Clusters
• Grid Computing
• Cloud Computing

When Do We Need HPC?

• HPC is essential for time-bound tasks, such as automotive design, weather prediction, or managing large websites.
• Automotive engineers need HPC to simulate scenarios that require extensive computational resources to develop fuel-efficient cars in a timely manner.
• Engineers at Amazon.com utilize HPC to manage the heavy traffic and large data quantities on websites to maintain user experience; to handle thousands of transactions per second.
• HPC is used in large-scale weather prediction agencies to determine weather patterns and other relevant data to create accurate forecasts.

What Does HPC Include?

• HPC encompasses high-speed computations in parallel, utilizing multiple compute elements (CPUs, GPUs), connected by a fast network.
• Hardware components include computer architecture (e.g., vector computers, distributed systems, clusters), network connections (e.g., InfiniBand, Ethernet), and proprietary networks.
• Software comprises programming models (e.g., MPI, SHMEM, PGAS) and applications (e.g., open-source, commercial software).

How Does HPC Work?

• High-performance computing solutions comprise compute, network, and storage components.
• HPC software programs and algorithms run simultaneously on servers within a cluster.
• The cluster network connects to data storage for output capture.
• These components function seamlessly to achieve complex tasks.

Parallel Architectures

• Traditional software functions in a serial manner, processing instructions sequentially on a single processor.
• Parallel computing utilizes multiple compute resources concurrently, breaking a problem into discrete parts for parallel execution; executing instructions simultaneously on various processors.

Parallel Architectures (Continued)

• Parallel computing involves executing multiple instructions simultaneously using multiple compute resources.
• A problem is broken down into smaller, parallel parts that can be executed concurrently.
• This parallel execution, using separate processors, is faster than serial computation, employing only one processor.

Parallel Architectures (Continued)

• Parallel architectures form the hardware foundation of HPC, using many processing units working simultaneously to solve complex tasks.
• Modern computers are inherently parallel, although there are distinct approaches: parallel architectures.
• There are multiple execution cores within a single processor (multi-cores) and networks connecting computers or parallel data processors (clusters and grids).

Parallel Architectures (Continued)

• Virtually all modern standalone computers are parallel in nature.
• These use many functional units (cache, branching, floating-point operations, and graphics processing units (GPUs)).
• A large number of processors enables quicker processing speed for parallel computation.
• Networks combine multiprocessors to perform larger parallel computation tasks.

Why Use Parallel Architectures?

• HPC save time & money by completing tasks quicker.
• Solutions can handle complex problems more easily.
• Non-local resources are utilized.
• Underlying parallel hardware is optimized and maximized.

Types of Parallelism

• Data parallelism involves distributing data across parallel computing nodes. One example is dividing a large matrix calculation between multiple CPU cores.
• Task parallelism distributes tasks among processors; for instance, splitting a complex task across various processors for faster execution.

Flynn's Taxonomy of Parallel Architectures

• Flynn's taxonomy categorizes parallel architectures based on instruction stream and data stream properties.
• SISD (Single Instruction, Single Data): only one instruction acts on one data element at a time (e.g., older mainframe computers).
• SIMD (Single Instruction, Multiple Data): the same instruction is applied to many distinct data elements simultaneously. (e.g., vector processors, GPUs).
• MISD (Multiple Instruction, Single Data): multiple instructions operate on the same data stream. This is rare.
• MIMD (Multiple Instruction, Multiple Data): Multiple instructions operate on multiple data, the standard for most modern computers (e.g., multi-core processors, clusters, networked PCs).

Parallel Computer Memory Architectures

• Shared-memory architectures (UMA, NUMA): processors share global address space to access memory.
• UMA (Uniform Memory Access): all processors have equal access time to memory.
• NUMA (Non-Uniform Memory Access): memory access varies depending on location, with potentially different access times.
• Distributed-memory architectures: each processor has its own memory, and processors need to communicate to share data.

Multi-Cores

• A multi-core processor involves multiple independent processing units—cores—within the CPU chip.
• Cores operate in parallel to enhance overall speed, enabling execution of concurrent instructions.

Why Multi-Cores?

• Utilizing multi-core technology increases the clock frequency limit for single-cores to allow for more complex tasks.
• The demand for multithreaded applications and more complex workloads is an important driver for the implementation of multi-core processors.

CPU Clocking

• Digital hardware operations are governed by a constant-rate clock.
• The clock period defines the duration of a clock cycle, while clock frequency indicates the number of cycles per second.

Multi-cores: Cache Coherence

• Cache coherence maintains consistency among multiple caches that share the same memory resources.
• Snooping and directory-based cache coherence mechanisms manage cache consistency, especially on multicore systems.

Multi-cores: Coherence Protocols

• Write-invalidate: When a core modifies data in a shared cache, other cores are notified, and their copies of the data are invalidated.
• Write-update: When a core modifies shared data, all other caching copies are updated simultaneously.

Graphical Processing Units (GPUs)

• GPUs excel at parallel processing, making them ideal for video, graphics, visual computing, and real-time computer graphics interaction with the calculated objects.
• GPU architecture differs from CPUs, featuring high parallelism, multithreading, and graphics image processing within a single chip.

GPU Evolution

• GPUs have evolved from simple VGA controllers to complex parallel processing units.
• Modern GPUs use CUDA (Compute Unified Device Architecture).
• GPUs are widely used in computing and visualization applications, due to their enhanced capabilities.

Why GPUs?

• GPUs offer a separate dedicated resource for graphics, freeing up the CPU for other tasks. They handle graphic operations more effectively by offloading the workload from the CPU in parallel.

GPU vs. CPU

• CPUs execute programs sequentially, while GPUs thrive on parallel operations.
• GPUs process instructions concurrently using multiple cores—a contrast to CPUs that function sequentially.
• GPUs offer significantly faster memory interfaces to achieve higher performance for graphics intensive applications.
• GPUs often use much deeper pipelines, accelerating parallel processing over traditional CPU pipelines.

Cloud Computing

• Cloud computing shifts computing away from individual devices to a cloud of computers. This concept abstracts away the server details for the client.
• Services are utilized with respect to the need when using cloud computing, freeing users from complex tasks.
• Virtualization in cloud environments creates numerous virtual instances by pooling and sharing resources, improving efficiency.

NIST Definition of Cloud Computing

• Cloud computing is a model for allowing universal, convenient access, on demand, to a pool of configurable resources (such as networks, services, etc.).

Characteristics of Cloud Computing

• On-demand self-service
• Broad network access
• Resource pooling
• Rapid elasticity
• Measured service

Cloud Components

• Clients
• Data Center
• Internet

Cloud Computing Benefits

• Lower Costs: Lower hardware, software costs via pay-as-you-go model.
• Quality of Service: Reliability, easy maintenance, simplified management.
• Ease of Utilization: Cloud resources are easily manageable and readily accessible.

Cloud Computing Limitations

• Reliance on a constant internet connection.
• Limitations for low-speed connections.
• Potential security and privacy risks.
• Possible service provider downtime concerns.
• Latency concerns.

Clusters

• A computer cluster comprises interconnected computers that function as a unified system, similar to a single computer. Tight or loose coupling differentiates cluster architectures depending on the task.
• High performance and reliability, with cost effectiveness and availability improvements.

Cluster Benefits

• High Availability, Performance, Low Cost, Elasticity.

Grid Computing

• Grid computing connects computers from different administrative domains to achieve a common objective (e.g., handling large datasets or performing complex simulations).
• It's a form of distributed computing where multiple loosely coupled computers share resources.

Types of Grid Computing: Data Grids

• Distributes data across a grid of computers, ensuring data coherency (consistency) among all the components.
• Replication: Copies data across all nodes.
• Invalidation: Data on one node is invalidated after changes to ensure data consistency.
• Distributed Transactions: Ensure consistency during changes across nodes.
• Data Backups: Handle backup and recovery processes.
• Data Affinity: Data-partitioning for assigning subsets to specific nodes.

Types of Grid Computing: Compute Grids

• Enables distributed execution of computations on multiple interconnected grid nodes.
• Improves scalability by offloading calculations.
• Advantages include automatic deployment, automatic node selection, and load balancing.

Grid Computing Advantages

• Solves large, complex problems quickly.
• Improved collaboration capabilities with other organizations.
• Leveraging existing hardware efficiently.

Grid Computing Disadvantages

• Complex grid-software implementation.
• Extensive setup and configuration challenges.
• Limited human interaction due to non-interactive nature of job submissions.

Beowulf Clusters

• Use parallel processing to create cost-effective, powerful supercomputers.
• Two main types of nodes: Master (or service) and worker nodes (or computing nodes).
• The master node manages and distributes tasks to worker nodes to achieve efficient problem-solving.

Clusters- Technologies to Implement

• Uses parallel virtual machines (PVMs) and Message Passing Interface (MPI).
• Providing environments for message-passing communications and managing tasks and resources that link nodes effectively.

Beowulf Clusters (Continued)

• Used in a variety of applications, like data processing and scientific simulations.

Test your knowledge on significant advancements in GPU technology, the differences between GPUs and CPUs, and key concepts in high-performance computing (HPC). This quiz covers topics from the introduction of single-chip graphics processors to parallel computing and multi-core processors.