Parallel Processing Unit 1 PDF

PARALLEL PROCESSING UNIT 1 1  UNDERSTANDING PARALLEL ENVIRONMENT 2 QUIZ What are 3 traditional ways HW Designers make computers run faster? Faster Clocks  Longer Clock Period More Work per Clock Cycle  Larger Hard Disk More Processors  Reduce amount of memory...

PARALLEL PROCESSING UNIT 1 1  UNDERSTANDING PARALLEL ENVIRONMENT 2 QUIZ What are 3 traditional ways HW Designers make computers run faster? Faster Clocks  Longer Clock Period More Work per Clock Cycle  Larger Hard Disk More Processors  Reduce amount of memory 3 SEYMOUR CRAY (SUPER COMPUTER DESIGNER)  Ifyou are plowing a field, which would you rather use?  Two strong oxen.  1024 chickens 4 PARALLEL COMPUTING  It was intended to be used by super computing.  Now all computers/mobiles are using parallel computing.  Modern GPUs  Hundred of processors  Thousand of ALUs (3,000)  Ten or thousands of concurrent threads.  This requires a different way of programming than a single scalar processor  General purpose programmability over GPU (GPGPU.) 5 TRANSISTORS CONTINUE ON MOORE’S PATH... FOR NOW 6 CLOCK SPEED (NO MORE SPEED) 7 QUIZ  Are processing today getting faster Because  We are clocking their transistors faster Wehave more transistors available for computation. o Why don’t we keep increasing clock speed of a single processor instead of multiprocessors with a less clock speed? o No, we can’t because of power (heat) 8 WHAT KIND OF PROCESSORS WILL WE BUILD?  Assume major design constraint is Power  Why are traditional CPU-like processors are not the most energy efficient processors?  It has complex control hardware  This increase flexibility and performance  And increase power consumption and design complexity as well  How to increase power efficiency (GPU-like)?  Build simple control structure.  Take those transistors and devote them to support more computation on the data path 9  The challenge becomes how to program? MORE TO UNDERSTAND 10  Less speed with MORE TO UNDERSTAND (CONT.) simple structure More speed with complex structure Less Power Power 11 QUIZ  Which techniques are computer designer using today to build more power-efficient chips?  Fewer, more complex processors More, Simpler processors  Maximizing the speed of the processor clock  Increasing the complexity of the control hardware 12 ANOTHER FACTOR FOR POWER EFFICIENCY Power Efficiency Decrease latency Increase Throughput (Amount of time to (Task completed per complete a task) unit time) “Time” “Number”  The two goals are not aligned  CPU-like: design to decrease latency  GPU-like: design to increase throughput 13  The choice depends on the application (Image processing prefer to increase the throughput) SUPER QUIZ  Why do I say GPU-like and not saying Multi-core CPU? Is there a deference ?!  They both build for parallel programming. However, Multi-core CPUs can be used for sequential and parallel programming as well (provides branches and interrupts ). On the other hand GPU build for parallel programming from scratch. 14 GPU DESIGN BELIEVES  Lots of simple compute units  Explicitly parallel programming model  We know there are many processors and we didn’t depend on the complier for example to parallel the task for us.  Optimized for throughput not latency 15