Parallel Processing Unit 1

Study Notes

Parallel Processing Unit 1

Parallel processing is a method of computation where multiple calculations are carried out simultaneously rather than sequentially.
Hardware designers use traditional methods to make computers faster, these include faster clocks, more work per clock cycle, and more processors.
Seymour Cray, a prominent supercomputer designer, exemplified the concept of parallel processing with the analogy of using oxen instead of chickens to plow a field. Oxen (multiple stronger resources) operate much more efficiently.
Modern computers and mobile devices now employ parallel processing.
Modern GPUs utilize hundreds of processors, including thousands of ALUs.
These devices utilize concurrent threads.
Programming for parallel systems differs significantly from programming traditional serial systems, requiring different programming techniques.
This shift towards parallel processing is driven by the limitations of scaling clock speed, as shown by Moore's Law related data about transistors.
Increasing clock speed was a traditional approach to boost computer speed, but this approach plateaus at a certain point, due to diminishing returns related to heat generation and power consumption, as opposed to utilizing more processors.
A key driving force behind parallel processing is the desire to increase throughput (completing more tasks per unit time) alongside maintaining low latency (reducing the time it takes to complete a single task).
GPU design emphasizes a parallel processing model and relies less on compiler intervention for parallel task execution, focusing on throughput.

Quiz Questions and Answers

Traditional methods to improve computer speed include: faster clocks, more work per clock cycle, and more processors.
Modern faster processing is a consequence of adding more processors and transistors, not making individual processors faster.
The current design constraint is power, not speed. CPUs are not as energy-efficient as alternative designs since they have complex control structures, increasing power consumption and design complexity.
To enhance power efficiency, GPU-like designs adopt simpler control structures, enabling better use of available transistors to focus on processing data paths.
The biggest challenge in these designs is effectively programming these structures for optimal performance.

Additional Points

Computer designers employ fewer, yet more complex processors.
Efficiency is achieved by maximizing the speed of the processors clock and simplifying the processor structure.
The two primary goals in design are not always aligned: decreasing latency (time) and increasing throughput (number of completed tasks).
For example, image processing applications prioritize higher throughput, while other types of applications might prioritize lower latency.