CAP 3 - Datapaths and Stalls: Control Hazards and Hardware Forwarding

Control hazards

A branch that changes the PC to the branch's target address

Redoing the fetch of the instruction following a branch

Predicted-taken scheme

Branch instructions

10-30%

The fetch and fall-through occur ordinarily

Execution continues with the instruction after the branch delay instruction

They use a single instruction delay

To always execute instructions in that slot

Heavily used

Not known any earlier than the branch outcome

To improve performance by guessing the outcome of branch conditions

Prediction is effective when the hit rates are high, such as in end-of-loop testing or searches

Each functional unit can be used only once per instruction

Speculative execution

To improve performance by allowing instructions to be executed while a branch is being resolved

Instructions must use functional units at the same stage as other instructions

To fetch the next instruction from the instruction memory and update the program counter (PC) register

To store the address of the next sequential instruction in memory

To store the instruction that will be worked on in subsequent clock cycles

Memory access/branch completion (MEM) cycle

Every MIPS instruction can be implemented in at most five clock cycles

Data transfer (DT)

The NPC register holds the address of the next sequential instruction in memory. It is incremented by 4 from the current program counter (PC) value during the instruction fetch (IF) cycle, so that it can be used to update the PC in the subsequent memory access (MEM) cycle.

The branch delay slot in the delayed branch scheme was used to improve performance by executing an instruction immediately following the branch instruction, even if the branch was taken. This helped to hide the latency of the branch instruction and keep the pipeline full.

Superscalar processors often use branch prediction techniques, such as the predicted-not-taken scheme, to speculate on the outcome of conditional branches. This allows the processor to continue executing instructions down the predicted path, avoiding pipeline stalls due to branch hazards.

The primary advantage of a pipelined processor over a multiple clock cycle processor is the ability to achieve higher throughput by overlapping the execution of multiple instructions. In a pipelined processor, instructions can be executed concurrently, allowing for greater instruction-level parallelism and improved overall performance.

The purpose of the instruction fetch (IF) cycle is to fetch the current instruction from the instruction memory (IMem) and load it into the instruction register (IR). During this cycle, the program counter (PC) is also incremented by 4 to point to the next sequential instruction in memory.

A one-cycle stall per branch in a five-stage pipeline can have a significant impact on performance, typically resulting in a performance loss in the range of 10-20%. This is because the branch stall causes a pipeline bubble, which reduces the overall throughput of the processor.

If the instruction in the branch delay slot is also a branch, it can lead to a cascade of branch delays, potentially causing significant performance degradation. This is because the execution of the second branch would also be delayed, creating a chain reaction that could stall the pipeline for multiple cycles.

In the predicted-not-taken scheme, the branch target address is not known until the branch outcome is verified during the ID cycle, whereas in the delayed branch scheme, the target address is known earlier, during the branch delay slot execution.

The delayed branch scheme addresses the performance impact of branches by executing the instruction in the branch delay slot regardless of whether the branch is taken or not, thereby reducing the number of pipeline stalls. However, this approach can lead to issues if the instruction in the delay slot is also a branch, potentially causing a cascade of branch delays and further performance degradation.

In the predicted-not-taken scheme, the branch target address is not known until the branch outcome is verified during the ID cycle, whereas in the delayed branch scheme, the target address is known earlier, during the branch delay slot execution. The predicted-not-taken scheme can lead to more pipeline stalls due to the delayed target address determination, while the delayed branch scheme can mitigate this issue but introduces the potential for a cascade of branch delays if the delay slot instruction is also a branch.

The delayed branch scheme addresses the performance impact of branches by executing the instruction in the branch delay slot regardless of whether the branch is taken or not, thereby reducing the number of pipeline stalls. However, this approach can lead to issues if the instruction in the delay slot is also a branch, potentially causing a cascade of branch delays and further performance degradation. In contrast, the predicted-not-taken scheme does not have this issue, but can lead to more pipeline stalls due to the delayed branch target address determination.

The predicted-not-taken scheme can lead to more pipeline stalls due to the delayed branch target address determination, while the delayed branch scheme can mitigate this issue by executing the instruction in the branch delay slot regardless of whether the branch is taken or not. However, the delayed branch scheme introduces the potential for a cascade of branch delays if the instruction in the delay slot is also a branch. This can cause significant performance degradation, as the execution of the second branch would also be delayed, creating a chain reaction that could stall the pipeline for multiple cycles.

In the predicted-not-taken scheme, the processor assumes that branch instructions will not be taken (i.e., the next sequential instruction will be executed). If the branch is not taken, no penalty is incurred. However, if the branch is taken, the pipeline must be flushed and refilled with instructions from the branch target address, resulting in a multi-cycle penalty.

In the delayed branch scheme, one or more instructions immediately following the branch instruction are executed, regardless of whether the branch is taken or not. These instructions are called branch delay slots. Early RISC processors utilized this scheme to improve performance by allowing useful work to be done while the branch was being resolved.

Control hazards arise from branching instructions, which can cause the processor to fetch and execute instructions from an incorrect path. If a branch is mispredicted, the entire pipeline may need to be flushed and refilled, resulting in a significant performance penalty. Data hazards, on the other hand, can often be resolved through techniques like forwarding or stalling, which have a relatively smaller impact on performance.

The predicted-not-taken scheme is simple but can incur significant penalties for taken branches. The predicted-taken scheme can improve performance for taken branches but may suffer penalties for untaken branches. The delayed branch scheme can improve performance by allowing useful work during branch resolution but requires compiler support and may not always have suitable instructions to fill the delay slots. Overall, there is a trade-off between performance, complexity, and branch prediction accuracy.

The instruction fetch (IF) cycle is the first stage of the pipeline, where the processor fetches the next instruction from memory using the program counter (PC). The fetched instruction is then passed to the subsequent pipeline stages for decoding, execution, memory access, and writeback. The IF stage must be coordinated with the other stages to ensure smooth and efficient pipeline operation.

Superscalar processors employ more advanced branch prediction techniques, such as branch target buffers (BTBs) and branch history tables (BHTs), to predict the outcome and target address of branches more accurately. These techniques leverage past branch behavior and can handle multiple branches in flight simultaneously, unlike the simpler schemes used in scalar processors.

The delayed branch technique introduces a delay slot after branch instructions where useful instructions can be placed to improve performance. However, compilers must carefully choose the instructions for this slot to ensure they are effectively executed regardless of whether the branch is taken or not. The tradeoff is between introducing this delay for all branches versus the potential performance gains from filling the delay slot with useful work.

Branch prediction involves guessing the outcome of a branch condition and proceeding execution as if that guess were correct. The key premise is that processor state cannot be affected by any misprediction. When predictions are often correct (high hit rates), execution can proceed without stalls, improving performance. Common cases with good prediction like loop branches and failed searches enable these performance gains.

In a multiple clock cycle processor, each functional unit can only be used once per instruction, with all instructions using the units in the same stage. A pipelined processor overlaps execution of instructions, with each stage handling a different instruction concurrently. This enables much higher throughput as multiple instructions are effectively executed in parallel, which is the key advantage of pipelining.

In the predicted-not-taken scheme, the processor continues executing sequentially after a branch. If the branch is untaken (falls through), no penalty is incurred as the next instructions are already being fetched. If the branch is taken, execution must stall while the target address is fetched and fed into the pipeline. The target address is only known after the branch condition is evaluated, incurring some delay on taken branches.

Branch prediction hit rates may be low in code with many unpredictable data-dependent branches, such as those dependent on complex calculations or input data. In these cases, past branch behavior provides little information to reliably predict future branches. Low hit rates mean frequent mispredictions, causing pipeline flushes and stalls that can severely degrade performance, potentially negating the benefits of branch prediction.

Superscalar processors can issue and execute multiple instructions per cycle, so they likely require more advanced branch prediction capabilities to keep multiple pipelines fed efficiently. For example, they may utilize more sophisticated predictors that can predict multiple branches per cycle, or predictors that integrate more complex heuristics and history tracking for better accuracy on harder-to-predict branches.