Parallel Processing Unit 1

Parallel Processing Unit 1

• Parallel processing is a method of computation where multiple calculations are carried out simultaneously rather than sequentially.
• Hardware designers use traditional methods to make computers faster, these include faster clocks, more work per clock cycle, and more processors.
• Seymour Cray, a prominent supercomputer designer, exemplified the concept of parallel processing with the analogy of using oxen instead of chickens to plow a field. Oxen (multiple stronger resources) operate much more efficiently.
• Modern computers and mobile devices now employ parallel processing.
• Modern GPUs utilize hundreds of processors, including thousands of ALUs.
• These devices utilize concurrent threads.
• Programming for parallel systems differs significantly from programming traditional serial systems, requiring different programming techniques.
• This shift towards parallel processing is driven by the limitations of scaling clock speed, as shown by Moore's Law related data about transistors.
• Increasing clock speed was a traditional approach to boost computer speed, but this approach plateaus at a certain point, due to diminishing returns related to heat generation and power consumption, as opposed to utilizing more processors.
• A key driving force behind parallel processing is the desire to increase throughput (completing more tasks per unit time) alongside maintaining low latency (reducing the time it takes to complete a single task).
• GPU design emphasizes a parallel processing model and relies less on compiler intervention for parallel task execution, focusing on throughput.

Quiz Questions and Answers

• Traditional methods to improve computer speed include: faster clocks, more work per clock cycle, and more processors.
• Modern faster processing is a consequence of adding more processors and transistors, not making individual processors faster.
• The current design constraint is power, not speed. CPUs are not as energy-efficient as alternative designs since they have complex control structures, increasing power consumption and design complexity.
• To enhance power efficiency, GPU-like designs adopt simpler control structures, enabling better use of available transistors to focus on processing data paths.
• The biggest challenge in these designs is effectively programming these structures for optimal performance.

Additional Points

• Computer designers employ fewer, yet more complex processors.
• Efficiency is achieved by maximizing the speed of the processors clock and simplifying the processor structure.
• The two primary goals in design are not always aligned: decreasing latency (time) and increasing throughput (number of completed tasks).
• For example, image processing applications prioritize higher throughput, while other types of applications might prioritize lower latency.