CAP7 - Computer Architecture Chapter 7

Flynn's Taxonomy

• Flynn's taxonomy classifies computer architectures into four categories: SISD, SIMD, MISD, and MIMD.
• SISD (Single Instruction, Single Data) is related to single processors, where one instruction works on one data at a time.
• SIMD (Single Instruction, Multiple Data) is related to vector architectures and graphics processing units, where one instruction works on multiple data at the same time.
• MISD (Multiple Instructions, Single Data) is related to fault tolerance, where different instructions handle one data.
• MIMD (Multiple Instructions, Multiple Data) is related to multiprocessors, where different instructions work on different data in parallel.

SIMD vs MIMD

• MIMD architecture needs to fetch one instruction per data operation, providing more flexibility.
• SIMD architecture is potentially more energy-efficient than MIMD, with a single instruction launching many data operations.
• In SIMD, the programmer thinks sequentially, achieving parallel speedup by having parallel data operations.

SIMD Vector Processors

• SIMD vector processors have high-level instructions operating on vectors, such as in equation (7.1).
• These processors are particularly useful for scientific and engineering applications, simulations, and multimedia applications.

Vector Processor Characteristics

• The parallelism of loops can be exposed by the programmer or compiler through the usage of vector instructions.
• The memory system is adapted to provide memory access to a whole vector instead of individual elements.
• Hardware only needs to check data hazards between two vector instructions once per vector operand.
• Control hazards are eliminated, and dependency verification logic is similar to scalar instructions.

Basic Architecture

• A vector processor consists of a scalar unit and vector units.
• The RISC-V vector instruction set extension (RV64V) has 32 vector registers and a scalar unit.
• Each vector register holds a single vector, and the vector register file provides a sufficient number of ports to feed all vector functional units.

Vector Instructions

• Examples of vector instructions include vadd, vsub, vdiv, and vld, which operate on vectors.
• These instructions perform operations on entire vectors, reducing the need for control hazards and dependency verification.

Flynn's Taxonomy

• Flynn's taxonomy classifies computer architectures into four categories: SISD, SIMD, MISD, and MIMD.
• SISD (Single Instruction, Single Data) is related to single processors, where one instruction works on one data at a time.
• SIMD (Single Instruction, Multiple Data) is related to vector architectures and graphics processing units, where one instruction works on multiple data at the same time.
• MISD (Multiple Instructions, Single Data) is related to fault tolerance, where different instructions handle one data.
• MIMD (Multiple Instructions, Multiple Data) is related to multiprocessors, where different instructions work on different data in parallel.

SIMD vs MIMD

• MIMD architecture needs to fetch one instruction per data operation, providing more flexibility.
• SIMD architecture is potentially more energy-efficient than MIMD, with a single instruction launching many data operations.
• In SIMD, the programmer thinks sequentially, achieving parallel speedup by having parallel data operations.

SIMD Vector Processors

• SIMD vector processors have high-level instructions operating on vectors, such as in equation (7.1).
• These processors are particularly useful for scientific and engineering applications, simulations, and multimedia applications.

Vector Processor Characteristics

• The parallelism of loops can be exposed by the programmer or compiler through the usage of vector instructions.
• The memory system is adapted to provide memory access to a whole vector instead of individual elements.
• Hardware only needs to check data hazards between two vector instructions once per vector operand.
• Control hazards are eliminated, and dependency verification logic is similar to scalar instructions.

Basic Architecture

• A vector processor consists of a scalar unit and vector units.
• The RISC-V vector instruction set extension (RV64V) has 32 vector registers and a scalar unit.
• Each vector register holds a single vector, and the vector register file provides a sufficient number of ports to feed all vector functional units.

Vector Instructions

• Examples of vector instructions include vadd, vsub, vdiv, and vld, which operate on vectors.
• These instructions perform operations on entire vectors, reducing the need for control hazards and dependency verification.

Vector Computers

• A vector loop for RV64V is shown in Eq.(7.1): ~ = a×X + Y, where X and Y are vectors of size n, and a is a scalar.
• This problem is known as the SAXPY (Single-precision a × X plus Y) or DAXPY (Double-precision a × X plus Y) loop, which forms the inner loop of the Linpack benchmark.

RV64G vs. RV64V

• The code in Listing 7.6 shows the implementation of the DAXPY loop using the RV64G ISA.
• The code in Listing 7.7 shows the implementation of the DAXPY loop using the RV64V ISA.
• The RV64V code has only 8 instructions, while the RV64G code has 258 instructions (32 iterations × 8 instructions + 2 setup instructions).
• The reduction in instructions is due to vector operations working on 32 elements.

Vector Instructions Optimizations

• Stalls in vector instructions are different from the RV64G ISA.
• In the RV64V code, an instruction stalls only for the first element in each vector, whereas in the RV64G code, every fadd.d instruction must wait for a fmul.d to avoid a RAW dependence.
• Chaining allows forwarding of element-dependent operations in vector operations.
• Convoys are sets of vector instructions that could execute together, without structural hazards or RAW hazards.

Vector Processors Architecture

• Modern vector computers have vector functional units with multiple parallel pipelines, i.e., lanes.
• Lanes are able to produce two or more results per clock cycle.
• Each lane contains one portion of the vector register file and one execution pipeline from each vector functional unit.

Vector-Length Registers

• A vector register's processor has a natural vector length determined by the maximum vector length (MVL).
• The vector-length register (VL) controls the length of any vector operation, including vector load/store.

Strip Mining

• Strip mining is a technique used when the vector size is unknown at compile time and may be greater than MVL.
• Strip mining consists of generating code where each vector operation is done for a size less than or equal to MVL.
• One loop handles any number of iterations that is a multiple of MVL, and another loop handles the remainder iteration, which must be less than the MVL size.

Vector Computers

• A vector loop for RV64V is shown in Eq.(7.1): ~ = a×X + Y, where X and Y are vectors of size n, and a is a scalar.
• This problem is known as the SAXPY (Single-precision a × X plus Y) or DAXPY (Double-precision a × X plus Y) loop, which forms the inner loop of the Linpack benchmark.

RV64G vs. RV64V

• The code in Listing 7.6 shows the implementation of the DAXPY loop using the RV64G ISA.
• The code in Listing 7.7 shows the implementation of the DAXPY loop using the RV64V ISA.
• The RV64V code has only 8 instructions, while the RV64G code has 258 instructions (32 iterations × 8 instructions + 2 setup instructions).
• The reduction in instructions is due to vector operations working on 32 elements.

Vector Instructions Optimizations

• Stalls in vector instructions are different from the RV64G ISA.
• In the RV64V code, an instruction stalls only for the first element in each vector, whereas in the RV64G code, every fadd.d instruction must wait for a fmul.d to avoid a RAW dependence.
• Chaining allows forwarding of element-dependent operations in vector operations.
• Convoys are sets of vector instructions that could execute together, without structural hazards or RAW hazards.

Vector Processors Architecture

• Modern vector computers have vector functional units with multiple parallel pipelines, i.e., lanes.
• Lanes are able to produce two or more results per clock cycle.
• Each lane contains one portion of the vector register file and one execution pipeline from each vector functional unit.

Vector-Length Registers

• A vector register's processor has a natural vector length determined by the maximum vector length (MVL).
• The vector-length register (VL) controls the length of any vector operation, including vector load/store.

Strip Mining

• Strip mining is a technique used when the vector size is unknown at compile time and may be greater than MVL.
• Strip mining consists of generating code where each vector operation is done for a size less than or equal to MVL.
• One loop handles any number of iterations that is a multiple of MVL, and another loop handles the remainder iteration, which must be less than the MVL size.

Graphics Processing Units (GPUs)

• GPUs are similar to a set of vector processors sharing hardware, with multiple SIMD processors acting as independent MIMD cores.
• The main difference between GPUs and vector processors is multithreading, which is fundamental to GPUs.

Programming the GPU

• NVIDIA developed a C-like programming language, CUDA, to program its GPUs.
• CUDA generates C/C++ code for the system processor (host) and a C/C++ dialect for the GPU (device).
• OpenCL is a CUDA-similar programming language, offering a vendor-independent language for multiple platforms.

CUDA Basics

• The CUDA thread is the lowest level of parallelism, following the paradigm of "single instruction, multiple threads" (SIMT).
• Threads are blocked together and executed in groups, with multithreaded SIMD hardware executing a whole block of threads.
• CUDA functions can have different modifiers, such as __device__, __global__, or __host__, to specify where they are executed and launched.

Variables and Function Calls

• CUDA variables can have modifiers, such as __device__, to allocate them to the GPU memory and make them accessible by all multithreaded SIMD processors.
• CUDA has an extended function call, the CUDA execution configuration, which specifies the dimensions of the code, thread blocks, and threads.

Specific Terms

• blockIdx is the identifier/index for blocks.
• threadIdx is the identifier/index of the current thread within its block.
• blockDim stands for the number of threads in a block, which comes from the dimBlock parameter.

Simple Example

• A conventional C code example can be converted to a CUDA version, which launches n threads, one per vector element, with 256 threads per thread block.
• The CUDA code computes the corresponding element index i based on the block ID, number of threads per block, and the thread ID.

Simple Example 2

• A grid (or vectorized loop) is composed of thread blocks, with each thread block processing up to 512 elements, i.e., 16 threads per block.
• The SIMD instruction executes 32 elements at a time, and the example has 16 thread blocks for 8,192 elements in the vectors.

Graphics Processing Units (GPUs)

• GPUs are similar to a set of vector processors sharing hardware, with multiple SIMD processors acting as independent MIMD cores.
• The main difference between GPUs and vector processors is multithreading, which is fundamental to GPUs.

Programming the GPU

• NVIDIA developed a C-like programming language, CUDA, to program its GPUs.
• CUDA generates C/C++ code for the system processor (host) and a C/C++ dialect for the GPU (device).
• OpenCL is a CUDA-similar programming language, offering a vendor-independent language for multiple platforms.

CUDA Basics

• The CUDA thread is the lowest level of parallelism, following the paradigm of "single instruction, multiple threads" (SIMT).
• Threads are blocked together and executed in groups, with multithreaded SIMD hardware executing a whole block of threads.
• CUDA functions can have different modifiers, such as __device__, __global__, or __host__, to specify where they are executed and launched.

Variables and Function Calls

• CUDA variables can have modifiers, such as __device__, to allocate them to the GPU memory and make them accessible by all multithreaded SIMD processors.
• CUDA has an extended function call, the CUDA execution configuration, which specifies the dimensions of the code, thread blocks, and threads.

Specific Terms

• blockIdx is the identifier/index for blocks.
• threadIdx is the identifier/index of the current thread within its block.
• blockDim stands for the number of threads in a block, which comes from the dimBlock parameter.

Simple Example

• A conventional C code example can be converted to a CUDA version, which launches n threads, one per vector element, with 256 threads per thread block.
• The CUDA code computes the corresponding element index i based on the block ID, number of threads per block, and the thread ID.

Simple Example 2

• A grid (or vectorized loop) is composed of thread blocks, with each thread block processing up to 512 elements, i.e., 16 threads per block.
• The SIMD instruction executes 32 elements at a time, and the example has 16 thread blocks for 8,192 elements in the vectors.