Parallel Computer Architectures Quiz

Parallel Computer Architectures

• Multiprocessor systems use multiple processors that execute instructions simultaneously, communicating through shared memory.

Flynn's Taxonomy

• Proposed by Michael J. Flynn in 1966
• A classification of parallel computer architectures.
• Categorized by the number of instruction and data streams.

Types of Architectures

• SISD (Single Instruction, Single Data):
  • A single processor executes a single instruction stream on a single data stream.
  • Instructions are broadcast to all processing elements.
  • Data is stored in a single memory.
  • A uni-processor is an example.
• SIMD (Single Instruction, Multiple Data):
  • A single instruction stream is executed on multiple data streams.
  • Processing elements (PEs) each have associated local memory.
  • Instructions are executed simultaneously on different data sets by different processors.
• MISD (Multiple Instruction, Single Data):
  • Multiple instructions are executed on a single data stream.
  • Less practical application.
• MIMD (Multiple Instruction, Multiple Data):
  • Multiple instructions are executed on multiple data streams simultaneously.
  • Processors can execute different instruction sequences on different data sets.
  • Can be shared memory (e.g., SMP, NUMA) or distributed memory (e.g., clusters).

MIMD - Shared Memory

• Processors share memory, communicating through shared memory.
• "Tightly coupled" system:
  • Single copy of the operating system.
  • Single address space.
  • Usually a single bus or backplane connecting processors and memories.
  • Very low communication latency.
• Different types:
  • SMP (Symmetric Multiprocessor):
    • Memory access time is approximately the same to different regions of memory.
  • NUMA (Non-uniform Memory Access):
    • Access times to different regions of memory may have different access times.

MIMD - Distributed Memory (Clusters)

• Collection of independent uniprocessors that run their own operating systems.
• Communication via a local area network.
• Often called nodes.
• Systems look differently based on host.
• Working together as a unified resource.
• Illusion of being one machine.
• Communication via fixed paths or network connections.

Cluster Benefits

• Absolute scalability: Can have tens, hundreds, or thousands of machines, exceeding the power of single machines.
• Incremental scalability: Cluster expansions can occur in small steps without major system upgrades.
• High availability: Failure of one node doesn't stop service as each node is a standalone computer.

Cluster Benefits (Cont.)

• Superior price/performance: Using commodity components creates cost effective solutions.

Memory Architecture

• Shared memory: Uniform Memory Access (UMA), Non-Uniform Memory Access (NUMA).
• Distributed memory: Often part of multiprocessor systems or clusters.

UMA Architecture

• Uniform memory access; identical processors have the same latency to access any memory location.
• Most commonly represented by Symmetric Multiprocessor (SMP) machines.
• Equal access & access times to memory.
• Cache coherency required.

NUMA Architecture

• Not all processors have the same access time to all memories.
• Can be made by physically linking two or more SMPs.
• Memory access across the link is slower.
• Each processor has its local memory.
• Memory of another processor is accessible but latency varies (remote memory access).

Fault Tolerance and Graceful Degradation

• Fault tolerance: Systems are designed so that a failed component has a backup.
• Graceful degradation: A system can continue to function with decreased abilities when parts fail, preventing complete breakdown.

Blade Servers

• Multiple processors, I/O, and networking boards in a single chassis.
• Each blade boots independently. Each runs its own operating system and application.

Beowulf Cluster

• Cluster implemented on multiple identical commercial off-the-shelf computers connected via a TCP/IP Ethernet local area network.

Massive Parallel Processing (MPP)

• A single computer with many networked processors.
• Large systems with "far more" than 100 processors.
• Each processor has its own memory and copy of the operating system and application.
• Communication using an interconnect network.

Grid Computing

• The most distributed form of parallel computing.
• Uses computers over the internet to solve a problem.
• Grid computing applications typically use Middleware to manage resources.
• Middleware like BOINC (Berkeley Open Infrastructure for Network Computing) is used for grid systems.

Test your knowledge on parallel computer architectures and Flynn's taxonomy. This quiz covers multiprocessor systems, types of architectures including SISD, SIMD, and MISD. Assess your understanding of how these systems operate and their classifications.