Podcast
Questions and Answers
What is the main characteristic of NUMA systems?
What is the main characteristic of NUMA systems?
- Each processor has its own cache, but no protocol for coherence
- Processors do not have cache and rely on main memory
- Memory is uniformly addressable from all processors, but access times vary (correct)
- Memory is not shared among processors
What is the primary difference between SMP and NUMA architectures?
What is the primary difference between SMP and NUMA architectures?
- NUMA has cache coherence protocol, SMP does not
- SMP has distributed memory, NUMA has shared memory
- SMP has uniform access time, NUMA does not (correct)
- SMP has shared memory, NUMA does not
In a cluster, what is the primary means of communication between processes?
In a cluster, what is the primary means of communication between processes?
- Distributed memory
- Cache coherence protocol
- Message passing (correct)
- Shared memory
What is the term for a computation consisting of setup, computation, and finalization sections?
What is the term for a computation consisting of setup, computation, and finalization sections?
What is the formula for the total running time of a program on one processing element?
What is the formula for the total running time of a program on one processing element?
What is the primary advantage of ccNUMA systems?
What is the primary advantage of ccNUMA systems?
What is the primary characteristic of distributed memory systems?
What is the primary characteristic of distributed memory systems?
What is the term for a group of off-the-shelf computers connected by a network?
What is the term for a group of off-the-shelf computers connected by a network?
What is the main motivation behind parallel computing?
What is the main motivation behind parallel computing?
What is the primary classification of parallel computers based on?
What is the primary classification of parallel computers based on?
What type of computer architecture is characterized by a single instruction stream operating on a single data stream?
What type of computer architecture is characterized by a single instruction stream operating on a single data stream?
What type of classification is mentioned in the context of parallel computers?
What type of classification is mentioned in the context of parallel computers?
What is the primary advantage of SIMD computers?
What is the primary advantage of SIMD computers?
What is the primary means of announcing course-related communications?
What is the primary means of announcing course-related communications?
Which type of computer architecture is characterized by multiple processors operating on multiple data items, each executing independent instructions?
Which type of computer architecture is characterized by multiple processors operating on multiple data items, each executing independent instructions?
What is the primary objective of understanding parallel computers and parallel computing?
What is the primary objective of understanding parallel computers and parallel computing?
What is the mode of operation in which the programmer starts up the same executable on the parallel processors?
What is the mode of operation in which the programmer starts up the same executable on the parallel processors?
What is the focus of the quantitative look at parallel computation?
What is the focus of the quantitative look at parallel computation?
What is the primary characteristic of shared memory systems?
What is the primary characteristic of shared memory systems?
What is the relevance of the course outline in the context of the course?
What is the relevance of the course outline in the context of the course?
Which type of computer architecture is characterized by multiple instruction streams operating on a single data stream?
Which type of computer architecture is characterized by multiple instruction streams operating on a single data stream?
What is the importance of regularly checking announcements on the ulwazi course site?
What is the importance of regularly checking announcements on the ulwazi course site?
What is the primary focus of the course COMS3008A?
What is the primary focus of the course COMS3008A?
What is the primary purpose of Flynn's taxonomy?
What is the primary purpose of Flynn's taxonomy?
Study Notes
Course Overview
- COMS3008A: Parallel Computing course by Hairong Bau
- Objectives:
- Understand basics of parallel computers and computing
- Understand motivation of parallel computing
- Learn classification of parallel computers
- Apply simple quantitative modeling for parallel program performance
Parallel Computing
- Parallel Computing: What is it?
- One class of shared-memory systems: Symmetric Multiprocessors (SMPs)
- Another class of shared-memory systems: Non-Uniform Memory Access (NUMA)
- Why parallelism?
- To increase processing power and speed
- Supercomputers
Classification of Parallel Computers
-
Flynn's taxonomy (1966) used for classification
-
Classification based on two independent dimensions:
- Instruction stream (Single/Multiple)
- Data stream (Single/Multiple)
-
Four types of parallel computers:
SISD (Single Instruction Stream Single Data Stream)
- Traditional CPU architecture
- Single instruction executed on single data item at a time
SIMD (Single Instruction Stream Multiple Data Stream)
- Multiple processors operating on different data items
- Excellent for array operations (e.g.,
for (i = 0; i < N; i++) a[i] = b[i] + c[i];)
MISD (Multiple Instruction Stream Single Data Stream)
- Each processing unit executes different instruction streams on single data stream
- Rarely used
MIMD (Multiple Instruction Stream Multiple Data Stream)
- Multiple processors operating on multiple data items
- Most current parallel computers are of this type
- Can operate in Single Program Multiple Data (SPMD) mode
- Further decomposed into:
- Shared memory systems (all processes share single address space)
- Distributed memory systems (each process has own address space, communicates via message passing)
Performance Modeling
- Simple performance modeling for parallel computing
- Computation consisting of three parts: setup, computation, and finalization
- Total running time on one processing element (PE):
Ttotal (1) = Tsetup + Tcompute + Tfinalization - Parallel computer with multiple PEs: ?
Studying That Suits You
Use AI to generate personalized quizzes and flashcards to suit your learning preferences.
Description
Explore the fundamentals of parallel computing, including its objectives, classification, and quantitative aspects. This course covers the basics of parallel computers, control structures, and more.
