Podcast Beta
Questions and Answers
What is the main characteristic of NUMA systems?
What is the primary difference between SMP and NUMA architectures?
In a cluster, what is the primary means of communication between processes?
What is the term for a computation consisting of setup, computation, and finalization sections?
Signup and view all the answers
What is the formula for the total running time of a program on one processing element?
Signup and view all the answers
What is the primary advantage of ccNUMA systems?
Signup and view all the answers
What is the primary characteristic of distributed memory systems?
Signup and view all the answers
What is the term for a group of off-the-shelf computers connected by a network?
Signup and view all the answers
What is the main motivation behind parallel computing?
Signup and view all the answers
What is the primary classification of parallel computers based on?
Signup and view all the answers
What type of computer architecture is characterized by a single instruction stream operating on a single data stream?
Signup and view all the answers
What type of classification is mentioned in the context of parallel computers?
Signup and view all the answers
What is the primary advantage of SIMD computers?
Signup and view all the answers
What is the primary means of announcing course-related communications?
Signup and view all the answers
Which type of computer architecture is characterized by multiple processors operating on multiple data items, each executing independent instructions?
Signup and view all the answers
What is the primary objective of understanding parallel computers and parallel computing?
Signup and view all the answers
What is the mode of operation in which the programmer starts up the same executable on the parallel processors?
Signup and view all the answers
What is the focus of the quantitative look at parallel computation?
Signup and view all the answers
What is the primary characteristic of shared memory systems?
Signup and view all the answers
What is the relevance of the course outline in the context of the course?
Signup and view all the answers
Which type of computer architecture is characterized by multiple instruction streams operating on a single data stream?
Signup and view all the answers
What is the importance of regularly checking announcements on the ulwazi course site?
Signup and view all the answers
What is the primary focus of the course COMS3008A?
Signup and view all the answers
What is the primary purpose of Flynn's taxonomy?
Signup and view all the answers
Study Notes
Course Overview
- COMS3008A: Parallel Computing course by Hairong Bau
- Objectives:
- Understand basics of parallel computers and computing
- Understand motivation of parallel computing
- Learn classification of parallel computers
- Apply simple quantitative modeling for parallel program performance
Parallel Computing
- Parallel Computing: What is it?
- One class of shared-memory systems: Symmetric Multiprocessors (SMPs)
- Another class of shared-memory systems: Non-Uniform Memory Access (NUMA)
- Why parallelism?
- To increase processing power and speed
- Supercomputers
Classification of Parallel Computers
-
Flynn's taxonomy (1966) used for classification
-
Classification based on two independent dimensions:
- Instruction stream (Single/Multiple)
- Data stream (Single/Multiple)
-
Four types of parallel computers:
SISD (Single Instruction Stream Single Data Stream)
- Traditional CPU architecture
- Single instruction executed on single data item at a time
SIMD (Single Instruction Stream Multiple Data Stream)
- Multiple processors operating on different data items
- Excellent for array operations (e.g.,
for (i = 0; i < N; i++) a[i] = b[i] + c[i];)
MISD (Multiple Instruction Stream Single Data Stream)
- Each processing unit executes different instruction streams on single data stream
- Rarely used
MIMD (Multiple Instruction Stream Multiple Data Stream)
- Multiple processors operating on multiple data items
- Most current parallel computers are of this type
- Can operate in Single Program Multiple Data (SPMD) mode
- Further decomposed into:
- Shared memory systems (all processes share single address space)
- Distributed memory systems (each process has own address space, communicates via message passing)
Performance Modeling
- Simple performance modeling for parallel computing
- Computation consisting of three parts: setup, computation, and finalization
- Total running time on one processing element (PE):
Ttotal (1) = Tsetup + Tcompute + Tfinalization - Parallel computer with multiple PEs: ?
Studying That Suits You
Use AI to generate personalized quizzes and flashcards to suit your learning preferences.
Description
Explore the fundamentals of parallel computing, including its objectives, classification, and quantitative aspects. This course covers the basics of parallel computers, control structures, and more.
