Parallel Hardware: Flynn's Taxonomy and Memory Access

In a SISD system, a single instruction stream is processed by a single processor. An example is a traditional desktop or laptop computer.

In a SIMD system, a single instruction is applied to multiple data streams simultaneously. An example is a GPU (Graphics Processing Unit).

In a MIMD system, multiple processors execute different instructions on different data. An example is a cluster of networked computers.

<p>In SPMD architecture, multiple processors execute the same program but on different data. An example is MPI (Message Passing Interface) based systems.</p> Signup and view all the answers

<p>MISD systems have multiple processors executing different instructions on the same data. An example is fault-tolerant systems that use multiple error-checking mechanisms.</p> Signup and view all the answers

<p>The maximum speedup for 32 processing units is 3.33, for 64 processing units is 5.71, and for 128 processing units is 10.</p> Signup and view all the answers

<p>Scaled speedup in parallel computing refers to the ratio of the execution time on a single processor to the execution time on multiple processors. It measures how efficiently an application can utilize additional processors.</p> Signup and view all the answers

<p>Gustafson’s Law states that as the number of processors increases, the problem size can be increased to keep the workload balanced, resulting in a speedup that grows linearly with the number of processors. It is used to determine the fraction of inherently sequential operations that can be tolerated to achieve a desired speedup.</p> Signup and view all the answers

<p>The maximum speedup predicted by Amdahl’s Law converges to a limit as the number of processors increases, due to the fixed fraction of serial work. In contrast, Gustafson’s Law allows for unbounded speedup as the number of processors increases by scaling the problem size.</p> Signup and view all the answers

<p>The peak achievable performance of a dot product of two vectors is 40MFLOPS, assuming 2 multiply-add units and considering memory fetches. If cache read time and CPU cycle are also counted, the rate is 38.64MFLOPS.</p> Signup and view all the answers

<p>The effective memory access time of one processor is 19.7MFLOPS. If the computation is memory-bound, the peak computation rate would be 19.7MFLOPS.</p> Signup and view all the answers

<p>Load balancing in parallel computing ensures that work is evenly distributed among processors, avoiding idle processors and maximizing resource utilization. It improves performance by minimizing execution time and preventing bottlenecks.</p> Signup and view all the answers