Instruction Pipelining

Basic Instruction Pipelining

• Execution of the next instruction begins before the completion of the previous one.
• Modern processors, microcontrollers, and CPUs utilize techniques to increase instruction throughput.

Instruction throughput is defined as the number of instructions that can be executed in a unit of time.

Instruction Pipelining Idea

• Splits CPU instruction processing into independent steps or operations based on the opcode.
• Allows the CPU's control logic to handle instructions at the processing rate of the slowest step.
• The slowest step of each instruction is much faster than the time needed to process the instruction as a single step.

Car Assembly Analogy

• Each step in car assembly carries out a single microinstruction/micro operation and is linked to another step.
• Each step is called a pipe stage, with stages connected to form a pipe.
• Instructions progress through stages and exit at the end.
• A processor cycle is the time needed to move an instruction one step down the pipeline.

Pipeline Stages Example

• RISC pipeline broken into five stages with flip flops between each stage as follows:

• Instruction Fetch (IF)

• Instruction Decode (ID)

• Execute (EX)

• Memory Access (MEM)

• Write-back (WB)

• Each instruction is processed through multiple stages at the same time

• A pipelined processor consists of internal modules that can work semi-independently on separate microinstructions.

• Stages are linked by flip-flops.

• Pipelining reduces the overall instruction processing time without reducing the number of stages.

Serial Processing

• Non-pipelined systems are less efficient, because some CPU modules remain idle while others are active during an instruction cycle.
• Pipelining does not completely remove idle time in a CPU.
• Increases in instruction throughput come from making CPU modules work in parallel.
• In a fully pipelined CPU, a new instruction can be accepted every clock cycle.

Quantitative Effects of Pipelining

• Time in non-seconds per instruction increases.
• Instructions take more cycles to execute, though the average CPI remains roughly the same.
• Clock speed increases.
• There is a decrease in total execution time, hence a lower average time per instruction

Pipelining Laundry Analogy

• Washing Clothes
• Washing: 30 min
• Drying: 40 min
• Ironing: 20 min
• Serial load time, where a student completes all their tasks before the next student can begin, is 6 hours for 4 students.
• This is derived from (30+40+20) * 4 = 6 hours.
• Pipelining shortens the load time to 3.5 hours for all four students.
• Speedup calculates how much faster pipelining is compared to serial processing

A speedup of 1.714 is ≈ 1.7 times faster.

Calculating total time in pipelining

• Let k be the number of stages in the pipeline, and tp be the time taken to execute per stage.
• Each instruction is a task T, and n is the number of tasks or instructions.
• First task requires k x tp time to complete in a k stage pipeline.
• The formula (k x tp) + (n - 1) tp = (k + n -1) tp applies if all stages take exactly the same time with no wait cycles.

Clock Skew and Latency

• Pipelining introduces clock skew, referring to different arrival times of clock signals, especially in adjacent flip-flops.
• In such cases, instruction latency is:

Latency = MAX(lengths of unpipelined stages) + overhead (clock skew).

• A non-pipelined machine with six execution stages of lengths 50 ns, 50 ns, 60 ns, 60 ns, 50 ns and 50 ns has an instruction latency of 320 ns and would take that amount of time * 100 to run 100 instructions.
• If pipelining is introduced with a clock skew of 5ns the instruction latency is 65ns and completing 100 instructions takes 6825ns.

Superscalar processors

• Execute more than one instruction in a cycle.
• They dispatch multiple instructions to several execution units.
• The execution unit is not a separate processor but is in the same processor; for example, an ALU.
• They implement instruction-level parallelism within a single processor.

Instruction pipelining increases throughput by starting the next instruction before the previous one finishes. It splits CPU instruction processing into independent steps, optimizing control logic. Each step in the process is called a pipe stage.