Computer Organization and Architecture PDF

CHAPTER A TOP-LEVEL VIEW OF COMPUTER FUNCTION AND INTERCONNECTION 3.1 Computer Components 3.2 Computer Function Instruction Fetch and Execute Interrupts I/O Function 3.3 Interconnection Structures 3.4 Bus Interconnection Bus St...

CHAPTER A TOP-LEVEL VIEW OF COMPUTER FUNCTION AND INTERCONNECTION 3.1 Computer Components 3.2 Computer Function Instruction Fetch and Execute Interrupts I/O Function 3.3 Interconnection Structures 3.4 Bus Interconnection Bus Structure Multiple-Bus Hierarchies Elements of Bus Design 3.5 PCI Bus Structure PCI Commands Data Transfers Arbitration 3.6 Recommended Reading and Web Sites 3.7 Key Terms, Review Questions, and Problems Appendix 3A Timing Diagrams 65 66 CHAPTER 3 / A TOP-LEVEL VIEW OF COMPUTER FUNCTION KEY POINTS ◆ An instruction cycle consists of an instruction fetch, followed by zero or more operand fetches, followed by zero or more operand stores, followed by an interrupt check (if interrupts are enabled). ◆ The major computer system components (processor, main memory, I/O modules) need to be interconnected in order to exchange data and control signals. The most popular means of interconnection is the use of a shared system bus consisting of multiple lines. In contemporary systems, there typ- ically is a hierarchy of buses to improve performance. ◆ Key design elements for buses include arbitration (whether permission to send signals on bus lines is controlled centrally or in a distributed fashion); timing (whether signals on the bus are synchronized to a central clock or are sent asynchronously based on the most recent transmission); and width (number of address lines and number of data lines). At a top level, a computer consists of CPU (central processing unit), memory, and I/O components, with one or more modules of each type. These components are intercon- nected in some fashion to achieve the basic function of the computer, which is to exe- cute programs.Thus, at a top level, we can describe a computer system by (1) describing the external behavior of each component—that is, the data and control signals that it exchanges with other components; and (2) describing the interconnection structure and the controls required to manage the use of the interconnection structure. This top-level view of structure and function is important because of its explana- tory power in understanding the nature of a computer. Equally important is its use to understand the increasingly complex issues of performance evaluation. A grasp of the top-level structure and function offers insight into system bottlenecks, alternate path- ways, the magnitude of system failures if a component fails, and the ease of adding per- formance enhancements. In many cases, requirements for greater system power and fail-safe capabilities are being met by changing the design rather than merely increas- ing the speed and reliability of individual components. This chapter focuses on the basic structures used for computer component in- terconnection. As background, the chapter begins with a brief examination of the basic components and their interface requirements. Then a functional overview is provided. We are then prepared to examine the use of buses to interconnect system components. 3.1 COMPUTER COMPONENTS As discussed in Chapter 2, virtually all contemporary computer designs are based on concepts developed by John von Neumann at the Institute for Advanced Studies, Princeton. Such a design is referred to as the von Neumann architecture and is based on three key concepts:  3.1 / COMPUTER COMPONENTS 67 Data and instructions are stored in a single read–write memory. The contents of this memory are addressable by location, without regard to the type of data contained there. Execution occurs in a sequential fashion (unless explicitly modified) from one instruction to the next. The reasoning behind these concepts was discussed in Chapter 2 but is worth summarizing here. There is a small set of basic logic components that can be com- bined in various ways to store binary data and to perform arithmetic and logical op- erations on that data. If there is a particular computation to be performed, a configuration of logic components designed specifically for that computation could be constructed. We can think of the process of connecting the various components in the desired configuration as a form of programming. The resulting “program” is in the form of hardware and is termed a hardwired program. Now consider this alternative. Suppose we construct a general-purpose config- uration of arithmetic and logic functions. This set of hardware will perform various functions on data depending on control signals applied to the hardware. In the orig- inal case of customized hardware, the system accepts data and produces results (Figure 3.1a). With general-purpose hardware, the system accepts data and control signals and produces results. Thus, instead of rewiring the hardware for each new program, the programmer merely needs to supply a new set of control signals. How shall control signals be supplied? The answer is simple but subtle. The en- tire program is actually a sequence of steps. At each step, some arithmetic or logical Sequence of Data arithmetic Results and logic functions (a) Programming in hardware Instruction Instruction codes interpreter Control signals General-purpose arithmetic Data Results and logic functions (b) Programming in software Figure 3.1 Hardware and Software Approaches 68 CHAPTER 3 / A TOP-LEVEL VIEW OF COMPUTER FUNCTION operation is performed on some data. For each step, a new set of control signals is needed. Let us provide a unique code for each possible set of control signals, and let us add to the general-purpose hardware a segment that can accept a code and gen- erate control signals (Figure 3.1b). Programming is now much easier. Instead of rewiring the hardware for each new program, all we need to do is provide a new sequence of codes. Each code is, in effect, an instruction, and part of the hardware interprets each instruction and gen- erates control signals. To distinguish this new method of programming, a sequence of codes or instructions is called software. Figure 3.1b indicates two major components of the system: an instruction in- terpreter and a module of general-purpose arithmetic and logic functions. These two constitute the CPU. Several other components are needed to yield a functioning computer. Data and instructions must be put into the system. For this we need some sort of input module. This module contains basic components for accepting data and instructions in some form and converting them into an internal form of signals us- able by the system. A means of reporting results is needed, and this is in the form of an output module. Taken together, these are referred to as I/O components. One more component is needed. An input device will bring instructions and data in sequentially. But a program is not invariably executed sequentially; it may jump around (e.g., the IAS jump instruction). Similarly, operations on data may re- quire access to more than just one element at a time in a predetermined sequence. Thus, there must be a place to store temporarily both instructions and data. That module is called memory, or main memory to distinguish it from external storage or peripheral devices. Von Neumann pointed out that the same memory could be used to store both instructions and data. Figure 3.2 illustrates these top-level components and suggests the interactions among them. The CPU exchanges data with memory. For this purpose, it typically makes use of two internal (to the CPU) registers: a memory address register (MAR), which specifies the address in memory for the next read or write, and a memory buffer register (MBR), which contains the data to be written into memory or receives the data read from memory. Similarly, an I/O address register (I/OAR) specifies a particular I/O device. An I/O buffer (I/OBR) register is used for the ex- change of data between an I/O module and the CPU. A memory module consists of a set of locations, defined by sequentially num- bered addresses. Each location contains a binary number that can be interpreted as either an instruction or data. An I/O module transfers data from external devices to CPU and memory, and vice versa. It contains internal buffers for temporarily hold- ing these data until they can be sent on. Having looked briefly at these major components, we now turn to an overview of how these components function together to execute programs. 3.2 COMPUTER FUNCTION The basic function performed by a computer is execution of a program, which con- sists of a set of instructions stored in memory. The processor does the actual work by executing instructions specified in the program. This section provides an overview of  3.2 / COMPUTER FUNCTION 69 CPU Main memory 0 System 1 bus 2 PC MAR Instruction Instruction Instruction IR MBR I/O AR Data Execution unit Data I/O BR Data Data I/O Module n–2 n–1 PC = Program counter Buffers IR = Instruction register MAR = Memory address register MBR = Memory buffer register I/O AR = Input/output address register I/O BR = Input/output buffer register Figure 3.2 Computer Components:Top-Level View the key elements of program execution. In its simplest form, instruction processing consists of two steps: The processor reads ( fetches) instructions from memory one at a time and executes each instruction. Program execution consists of repeating the process of instruction fetch and instruction execution. The instruction execution may involve several operations and depends on the nature of the instruction (see, for example, the lower portion of Figure 2.4). The processing required for a single instruction is called an instruction cycle. Using the simplified two-step description given previously, the instruction cycle is de- picted in Figure 3.3. The two steps are referred to as the fetch cycle and the execute cycle. Program execution halts only if the machine is turned off, some sort of unrecov- erable error occurs, or a program instruction that halts the computer is encountered. Instruction Fetch and Execute At the beginning of each instruction cycle, the processor fetches an instruction from memory. In a typical processor, a register called the program counter (PC) holds the address of the instruction to be fetched next. Unless told otherwise, the processor 70 CHAPTER 3 / A TOP-LEVEL VIEW OF COMPUTER FUNCTION Fetch cycle Execute cycle Fetch next Execute START instruction instruction HALT Figure 3.3 Basic Instruction Cycle always increments the PC after each instruction fetch so that it will fetch the next in- struction in sequence (i.e., the instruction located at the next higher memory ad- dress). So, for example, consider a computer in which each instruction occupies one 16-bit word of memory. Assume that the program counter is set to location 300. The processor will next fetch the instruction at location 300. On succeeding instruction cycles, it will fetch instructions from locations 301, 302, 303, and so on. This sequence may be altered, as explained presently. The fetched instruction is loaded into a register in the processor known as the instruction register (IR). The instruction contains bits that specify the action the processor is to take. The processor interprets the instruction and performs the re- quired action. In general, these actions fall into four categories: Processor-memory: Data may be transferred from processor to memory or from memory to processor. Processor-I/O: Data may be transferred to or from a peripheral device by transferring between the processor and an I/O module. Data processing: The processor may perform some arithmetic or logic opera- tion on data. Control: An instruction may specify that the sequence of execution be altered. For example, the processor may fetch an instruction from location 149, which specifies that the next instruction be from location 182. The processor will re- member this fact by setting the program counter to 182. Thus, on the next fetch cycle, the instruction will be fetched from location 182 rather than 150. An instruction’s execution may involve a combination of these actions. Consider a simple example using a hypothetical machine that includes the characteristics listed in Figure 3.4. The processor contains a single data register, called an accumulator (AC). Both instructions and data are 16 bits long. Thus, it is convenient to organize memory using 16-bit words. The instruction format provides 4 bits for the opcode, so that there can be as many as 24 = 16 different opcodes, and up to 212 = 4096 (4K) words of memory can be directly addressed. Figure 3.5 illustrates a partial program execution, showing the relevant por- tions of memory and processor registers.1 The program fragment shown adds the contents of the memory word at address 940 to the contents of the memory word at 1 Hexadecimal notation is used, in which each digit represents 4 bits. This is the most convenient notation for representing the contents of memory and registers when the word length is a multiple of 4. See Chap- ter 19 for a basic refresher on number systems (decimal, binary, hexadecimal).  3.2 / COMPUTER FUNCTION 71 0 3 4 15 Opcode Address (a) Instruction format 0 1 15 Magnitude (b) Integer format Program counter (PC)  Address of instruction Instruction register (IR)  Instruction being executed Accumulator (AC)  Temporary storage (c) Internal CPU registers 0001  Load AC from memory 0010  Store AC to memory 0101  Add to AC from memory (d) Partial list of opcodes Figure 3.4 Characteristics of a Hypothetical Machine Memory CPU registers Memory CPU registers 300 1 9 4 0 3 0 0 PC 300 1 9 4 0 3 0 1 PC 301 5 9 4 1 AC 301 5 9 4 1 0 0 0 3 AC 302 2 9 4 1 1 9 4 0 IR 302 2 9 4 1 1 9 4 0 IR 940 0 0 0 3 940 0 0 0 3 941 0 0 0 2 941 0 0 0 2 Step 1 Step 2 Memory CPU registers Memory CPU registers 300 1 9 4 0 3 0 1 PC 300 1 9 4 0 3 0 2 PC 301 5 9 4 1 0 0 0 3 AC 301 5 9 4 1 0 0 0 5 AC 302 2 9 4 1 5 9 4 1 IR 302 2 9 4 1 5 9 4 1 IR 940 0 0 0 3 940 0 0 0 3 325 941 0 0 0 2 941 0 0 0 2 Step 3 Step 4 Memory CPU registers Memory CPU registers 300 1 9 4 0 3 0 2 PC 300 1 9 4 0 3 0 3 PC 301 5 9 4 1 0 0 0 5 AC 301 5 9 4 1 0 0 0 5 AC 302 2 9 4 1 2 9 4 1 IR 302 2 9 4 1 2 9 4 1 IR 940 0 0 0 3 940 0 0 0 3 941 0 0 0 2 941 0 0 0 5 Step 5 Step 6 Figure 3.5 Example of Program Execution (contents of memory and registers in hexadecimal) 72 CHAPTER 3 / A TOP-LEVEL VIEW OF COMPUTER FUNCTION address 941 and stores the result in the latter location. Three instructions, which can be described as three fetch and three execute cycles, are required: 1. The PC contains 300, the address of the first instruction. This instruction (the value 1940 in hexadecimal) is loaded into the instruction register IR and the PC is incremented. Note that this process involves the use of a memory ad- dress register (MAR) and a memory buffer register (MBR). For simplicity, these intermediate registers are ignored. 2. The first 4 bits (first hexadecimal digit) in the IR indicate that the AC is to be loaded. The remaining 12 bits (three hexadecimal digits) specify the address (940) from which data are to be loaded. 3. The next instruction (5941) is fetched from location 301 and the PC is incremented. 4. The old contents of the AC and the contents of location 941 are added and the result is stored in the AC. 5. The next instruction (2941) is fetched from location 302 and the PC is incremented. 6. The contents of the AC are stored in location 941. In this example, three instruction cycles, each consisting of a fetch cycle and an execute cycle, are needed to add the contents of location 940 to the contents of 941. With a more complex set of instructions, fewer cycles would be needed. Some older processors, for example, included instructions that contain more than one memory address. Thus the execution cycle for a particular instruction on such processors could involve more than one reference to memory. Also, instead of memory refer- ences, an instruction may specify an I/O operation. For example, the PDP-11 processor includes an instruction, expressed symbol- ically as ADD B,A, that stores the sum of the contents of memory locations B and A into memory location A. A single instruction cycle with the following steps occurs: Fetch the ADD instruction. Read the contents of memory location A into the processor. Read the contents of memory location B into the processor. In order that the contents of A are not lost, the processor must have at least two registers for storing memory values, rather than a single accumulator. Add the two values. Write the result from the processor to memory location A. Thus, the execution cycle for a particular instruction may involve more than one reference to memory. Also, instead of memory references, an instruction may specify an I/O operation. With these additional considerations in mind, Figure 3.6 provides a more detailed look at the basic instruction cycle of Figure 3.3. The figure is in the form of a state diagram. For any given instruction cycle, some states may be null and others may be visited more than once.The states can be described as follows: Instruction address calculation (iac): Determine the address of the next in- struction to be executed. Usually, this involves adding a fixed number to the  3.2 / COMPUTER FUNCTION 73 Instruction Operand Operand fetch fetch store Multiple Multiple operands results Instruction Instruction Operand Operand Data address operation address address operation calculation decoding calculation calculation Instruction complete, Return for string fetch next instruction or vector data Figure 3.6 Instruction Cycle State Diagram address of the previous instruction. For example, if each instruction is 16 bits long and memory is organized into 16-bit words, then add 1 to the previous ad- dress. If, instead, memory is organized as individually addressable 8-bit bytes, then add 2 to the previous address. Instruction fetch (if): Read instruction from its memory location into the processor. Instruction operation decoding (iod): Analyze instruction to determine type of operation to be performed and operand(s) to be used. Operand address calculation (oac): If the operation involves reference to an operand in memory or available via I/O, then determine the address of the operand. Operand fetch (of): Fetch the operand from memory or read it in from I/O. Data operation (do): Perform the operation indicated in the instruction. Operand store (os): Write the result into memory or out to I/O. States in the upper part of Figure 3.6 involve an exchange between the processor and either memory or an I/O module. States in the lower part of the diagram involve only internal processor operations.The oac state appears twice, because an instruction may involve a read, a write, or both. However, the action performed during that state is fundamentally the same in both cases, and so only a single state identifier is needed. Also note that the diagram allows for multiple operands and multiple results, because some instructions on some machines require this. For example, the PDP-11 instruction ADD A,B results in the following sequence of states: iac, if, iod, oac, of, oac, of, do, oac, os. Finally, on some machines, a single instruction can specify an operation to be per- formed on a vector (one-dimensional array) of numbers or a string (one-dimensional array) of characters. As Figure 3.6 indicates, this would involve repetitive operand fetch and/or store operations. 74 CHAPTER 3 / A TOP-LEVEL VIEW OF COMPUTER FUNCTION Table 3.1 Classes of Interrupts Program Generated by some condition that occurs as a result of an instruction execution, such as arithmetic overflow, division by zero, attempt to execute an illegal machine instruction, or reference outside a user’s allowed memory space. Timer Generated by a timer within the processor. This allows the operating system to perform certain functions on a regular basis. I/O Generated by an I/O controller, to signal normal completion of an operation or to signal a variety of error conditions. Hardware failure Generated by a failure such as power failure or memory parity error. Interrupts Virtually all computers provide a mechanism by which other modules (I/O, mem- ory) may interrupt the normal processing of the processor. Table 3.1 lists the most common classes of interrupts. The specific nature of these interrupts is examined later in this book, especially in Chapters 7 and 12. However, we need to introduce the concept now to understand more clearly the nature of the instruction cycle and the implications of interrupts on the interconnection structure. The reader need not be concerned at this stage about the details of the generation and processing of in- terrupts, but only focus on the communication between modules that results from interrupts. Interrupts are provided primarily as a way to improve processing efficiency. For example, most external devices are much slower than the processor. Suppose that the processor is transferring data to a printer using the instruction cycle scheme of Figure 3.3. After each write operation, the processor must pause and remain idle until the printer catches up. The length of this pause may be on the order of many hundreds or even thousands of instruction cycles that do not involve memory. Clearly, this is a very wasteful use of the processor. Figure 3.7a illustrates this state of affairs. The user program performs a series of WRITE calls interleaved with processing. Code segments 1, 2, and 3 refer to se- quences of instructions that do not involve I/O. The WRITE calls are to an I/O pro- gram that is a system utility and that will perform the actual I/O operation. The I/O program consists of three sections: A sequence of instructions, labeled 4 in the figure, to prepare for the actual I/O operation. This may include copying the data to be output into a special buffer and preparing the parameters for a device command. The actual I/O command. Without the use of interrupts, once this command is issued, the program must wait for the I/O device to perform the requested function (or periodically poll the device). The program might wait by simply repeatedly performing a test operation to determine if the I/O operation is done. A sequence of instructions, labeled 5 in the figure, to complete the opera- tion. This may include setting a flag indicating the success or failure of the operation.  User I/O User I/O User I/O program program program program program program 1 4 1 4 1 4 I/O I/O I/O command command command WRITE WRITE WRITE 5 2a END 2 2 Interrupt Interrupt 2b handler handler WRITE WRITE 5 WRITE 5 END END 3a 3 3 3b WRITE WRITE WRITE (a) No interrupts (b) Interrupts; short I/O wait (c) Interrupts; long I/O wait Figure 3.7 Program Flow of Control without and with Interrupts 75 76 CHAPTER 3 / A TOP-LEVEL VIEW OF COMPUTER FUNCTION Because the I/O operation may take a relatively long time to complete, the I/O program is hung up waiting for the operation to complete; hence, the user program is stopped at the point of the WRITE call for some considerable period of time. INTERRUPTS AND THE INSTRUCTION CYCLE With interrupts, the processor can be engaged in executing other instructions while an I/O operation is in progress. Consider the flow of control in Figure 3.7b. As before, the user program reaches a point at which it makes a system call in the form of a WRITE call. The I/O program that is invoked in this case consists only of the preparation code and the actual I/O command. After these few instructions have been executed, control returns to the user program. Meanwhile, the external device is busy accepting data from computer memory and printing it. This I/O operation is conducted concurrently with the exe- cution of instructions in the user program. When the external device becomes ready to be serviced—that is, when it is ready to accept more data from the processor,—the I/O module for that external device sends an interrupt request signal to the processor. The processor responds by suspending operation of the current program, branching off to a program to service that particular I/O device, known as an interrupt handler, and resuming the original execution after the device is serviced. The points at which such interrupts occur are indicated by an asterisk in Figure 3.7b. From the point of view of the user program, an interrupt is just that: an inter- ruption of the normal sequence of execution. When the interrupt processing is com- pleted, execution resumes (Figure 3.8). Thus, the user program does not have to contain any special code to accommodate interrupts; the processor and the operat- ing system are responsible for suspending the user program and then resuming it at the same point. To accommodate interrupts, an interrupt cycle is added to the instruction cycle, as shown in Figure 3.9. In the interrupt cycle, the processor checks to see if any User program Interrupt handler 1 2 i Interrupt occurs here i1 M Figure 3.8 Transfer of Control via Interrupts  3.2 / COMPUTER FUNCTION 77 Fetch cycle Execute cycle Interrupt cycle Interrupts disabled Check for Fetch next Execute START instruction instruction interrupt; Interrupts process interrupt enabled HALT Figure 3.9 Instruction Cycle with Interrupts interrupts have occurred, indicated by the presence of an interrupt signal. If no interrupts are pending, the processor proceeds to the fetch cycle and fetches the next instruction of the current program. If an interrupt is pending, the processor does the following: It suspends execution of the current program being executed and saves its context. This means saving the address of the next instruction to be executed (current contents of the program counter) and any other data relevant to the processor’s current activity. It sets the program counter to the starting address of an interrupt handler routine. The processor now proceeds to the fetch cycle and fetches the first instruction in the interrupt handler program, which will service the interrupt. The interrupt han- dler program is generally part of the operating system. Typically, this program deter- mines the nature of the interrupt and performs whatever actions are needed. In the example we have been using, the handler determines which I/O module generated the interrupt and may branch to a program that will write more data out to that I/O module. When the interrupt handler routine is completed, the processor can resume execution of the user program at the point of interruption. It is clear that there is some overhead involved in this process. Extra instructions must be executed (in the interrupt handler) to determine the nature of the interrupt and to decide on the appropriate action. Nevertheless, because of the relatively large amount of time that would be wasted by simply waiting on an I/O operation, the processor can be employed much more efficiently with the use of interrupts. To appreciate the gain in efficiency, consider Figure 3.10, which is a timing dia- gram based on the flow of control in Figures 3.7a and 3.7b. Figures 3.7b and 3.10 as- sume that the time required for the I/O operation is relatively short: less than the time to complete the execution of instructions between write operations in the user program. The more typical case, especially for a slow device such as a printer, is that the I/O operation will take much more time than executing a sequence of user in- structions. Figure 3.7c indicates this state of affairs. In this case, the user program reaches the second WRITE call before the I/O operation spawned by the first call is 78 CHAPTER 3 / A TOP-LEVEL VIEW OF COMPUTER FUNCTION Time 1 1 4 4 Processor I/O I/O 2a wait operation operation 5 5 2b 2 4 I/O 4 3a operation Processor I/O 5 wait operation 5 3b (b) With interrupts 3 (a) Without interrupts Figure 3.10 Program Timing: Short I/O Wait complete. The result is that the user program is hung up at that point. When the preceding I/O operation is completed, this new WRITE call may be processed, and a new I/O operation may be started. Figure 3.11 shows the timing for this situation with and without the use of interrupts. We can see that there is still a gain in effi- ciency because part of the time during which the I/O operation is underway over- laps with the execution of user instructions. Figure 3.12 shows a revised instruction cycle state diagram that includes inter- rupt cycle processing. MULTIPLE INTERRUPTS The discussion so far has focused only on the occur- rence of a single interrupt. Suppose, however, that multiple interrupts can occur. For example, a program may be receiving data from a communications line and printing results. The printer will generate an interrupt every time that it com- pletes a print operation. The communication line controller will generate an in- terrupt every time a unit of data arrives. The unit could either be a single character or a block, depending on the nature of the communications discipline.  3.2 / COMPUTER FUNCTION 79 Time 1 1 4 4 Processor I/O 2 wait operation I/O operation Processor 5 wait 5 2 4 4 3 Processor I/O I/O wait operation operation Processor wait 5 5 (b) With interrupts 3 (a) Without interrupts Figure 3.11 Program Timing: Long I/O Wait In any case, it is possible for a communications interrupt to occur while a printer interrupt is being processed. Two approaches can be taken to dealing with multiple interrupts. The first is to disable interrupts while an interrupt is being processed. A disabled interrupt simply means that the processor can and will ignore that interrupt request signal. If an inter- rupt occurs during this time, it generally remains pending and will be checked by the processor after the processor has enabled interrupts.Thus, when a user program is exe- cuting and an interrupt occurs, interrupts are disabled immediately. After the interrupt handler routine completes, interrupts are enabled before resuming the user program, and the processor checks to see if additional interrupts have occurred.This approach is nice and simple, as interrupts are handled in strict sequential order (Figure 3.13a). 80 Instruction Operand Operand fetch fetch store Multiple Multiple operands results Instruction Instruction Operand Operand Data Interrupt address operation address address Interrupt operation check calculation decoding calculation calculation Instruction complete, Return for string No fetch next instruction or vector data interrupt Figure 3.12 Instruction Cycle State Diagram, with Interrupts  Interrupt User program handler X Interrupt handler Y (a) Sequential interrupt processing Interrupt User program handler X Interrupt handler Y (b) Nested interrupt processing Figure 3.13 Transfer of Control with Multiple Interrupts 81 82 CHAPTER 3 / A TOP-LEVEL VIEW OF COMPUTER FUNCTION Printer Communication interrupt interrupt User program service routine service routine t0 15 10 t t t  25 Disk t interrupt 40 t2 5 service routine t 35 Figure 3.14 Example Time Sequence of Multiple Interrupts The drawback to the preceding approach is that it does not take into account relative priority or time-critical needs. For example, when input arrives from the communications line, it may need to be absorbed rapidly to make room for more input. If the first batch of input has not been processed before the second batch arrives, data may be lost. A second approach is to define priorities for interrupts and to allow an interrupt of higher priority to cause a lower-priority interrupt handler to be itself interrupted (Figure 3.13b). As an example of this second approach, consider a system with three I/O devices: a printer, a disk, and a communications line, with increasing priorities of 2, 4, and 5, respectively. Figure 3.14, based on an example in [TANE97], illustrates a pos- sible sequence. A user program begins at t = 0. At t = 10, a printer interrupt occurs; user information is placed on the system stack and execution continues at the printer interrupt service routine (ISR). While this routine is still executing, at t = 15, a com- munications interrupt occurs. Because the communications line has higher priority than the printer, the interrupt is honored. The printer ISR is interrupted, its state is pushed onto the stack, and execution continues at the communications ISR.While this routine is executing, a disk interrupt occurs (t = 20). Because this interrupt is of lower priority, it is simply held, and the communications ISR runs to completion. When the communications ISR is complete (t = 25), the previous processor state is restored, which is the execution of the printer ISR. However, before even a single instruction in that routine can be executed, the processor honors the higher- priority disk interrupt and control transfers to the disk ISR. Only when that routine is  3.3 / INTERCONNECTION STRUCTURES 83 complete (t = 35) is the printer ISR resumed. When that routine completes (t = 40), control finally returns to the user program. I/O Function Thus far, we have discussed the operation of the computer as controlled by the processor, and we have looked primarily at the interaction of processor and mem- ory. The discussion has only alluded to the role of the I/O component. This role is discussed in detail in Chapter 7, but a brief summary is in order here. An I/O module (e.g., a disk controller) can exchange data directly with the processor. Just as the processor can initiate a read or write with memory, designat- ing the address of a specific location, the processor can also read data from or write data to an I/O module. In this latter case, the processor identifies a specific device that is controlled by a particular I/O module. Thus, an instruction sequence similar in form to that of Figure 3.5 could occur, with I/O instructions rather than memory- referencing instructions. In some cases, it is desirable to allow I/O exchanges to occur directly with memory. In such a case, the processor grants to an I/O module the authority to read from or write to memory, so that the I/O-memory transfer can occur without tying up the processor. During such a transfer, the I/O module issues read or write com- mands to memory, relieving the processor of responsibility for the exchange. This operation is known as direct memory access (DMA) and is examined Chapter 7. 3.3 INTERCONNECTION STRUCTURES A computer consists of a set of components or modules of three basic types (proces- sor, memory, I/O) that communicate with each other. In effect, a computer is a net- work of basic modules. Thus, there must be paths for connecting the modules. The collection of paths connecting the various modules is called the interconnection structure. The design of this structure will depend on the exchanges that must be made among modules. Figure 3.15 suggests the types of exchanges that are needed by indicating the major forms of input and output for each module type:2 Memory: Typically, a memory module will consist of N words of equal length. Each word is assigned a unique numerical address (0, 1,... , N – 1). A word of data can be read from or written into the memory. The nature of the operation is indicated by read and write control signals. The location for the operation is specified by an address. I/O module: From an internal (to the computer system) point of view, I/O is functionally similar to memory. There are two operations, read and write. Fur- ther, an I/O module may control more than one external device. We can refer to each of the interfaces to an external device as a port and give each a unique address (e.g., 0, 1,... , M – 1). In addition, there are external data paths for the 2 The wide arrows represent multiple signal lines carrying multiple bits of information in parallel. Each narrow arrows represents a single signal line. 84 CHAPTER 3 / A TOP-LEVEL VIEW OF COMPUTER FUNCTION Read Memory Write N words Address 0 Data Data N–1 Read I/O module Internal Write data External Address M ports data Internal data Interrupt signals External data Instructions Address Control Data CPU signals Interrupt Data signals Figure 3.15 Computer Modules input and output of data with an external device. Finally, an I/O module may be able to send interrupt signals to the processor. Processor: The processor reads in instructions and data, writes out data after processing, and uses control signals to control the overall operation of the sys- tem. It also receives interrupt signals. The preceding list defines the data to be exchanged. The interconnection structure must support the following types of transfers: Memory to processor: The processor reads an instruction or a unit of data from memory. Processor to memory: The processor writes a unit of data to memory. I/O to processor: The processor reads data from an I/O device via an I/O module. Processor to I/O: The processor sends data to the I/O device. I/O to or from memory: For these two cases, an I/O module is allowed to ex- change data directly with memory, without going through the processor, using direct memory access (DMA).  3.4 / BUS INTERCONNECTION 85 Over the years, a number of interconnection structures have been tried. By far the most common is the bus and various multiple-bus structures. The remainder of this chapter is devoted to an assessment of bus structures. 3.4 BUS INTERCONNECTION A bus is a communication pathway connecting two or more devices. A key charac- teristic of a bus is that it is a shared transmission medium. Multiple devices connect to the bus, and a signal transmitted by any one device is available for reception by all other devices attached to the bus. If two devices transmit during the same time pe- riod, their signals will overlap and become garbled. Thus, only one device at a time can successfully transmit. Typically, a bus consists of multiple communication pathways, or lines. Each line is capable of transmitting signals representing binary 1 and binary 0. Over time, a sequence of binary digits can be transmitted across a single line. Taken together, several lines of a bus can be used to transmit binary digits simultaneously (in paral- lel). For example, an 8-bit unit of data can be transmitted over eight bus lines. Computer systems contain a number of different buses that provide pathways between components at various levels of the computer system hierarchy. A bus that connects major computer components (processor, memory, I/O) is called a system bus. The most common computer interconnection structures are based on the use of one or more system buses. Bus Structure A system bus consists, typically, of from about 50 to hundreds of separate lines. Each line is assigned a particular meaning or function. Although there are many different bus designs, on any bus the lines can be classified into three functional groups (Figure 3.16): data, address, and control lines. In addition, there may be power distri- bution lines that supply power to the attached modules. The data lines provide a path for moving data among system modules. These lines, collectively, are called the data bus. The data bus may consist of 32, 64, 128, or even more separate lines, the number of lines being referred to as the width of the data bus. Because each line can carry only 1 bit at a time, the number of lines deter- mines how many bits can be transferred at a time. The width of the data bus is a key CPU Memory Memory I/O I/O Control lines Address lines Bus Data lines Figure 3.16 Bus Interconnection Scheme 86 CHAPTER 3 / A TOP-LEVEL VIEW OF COMPUTER FUNCTION factor in determining overall system performance. For example, if the data bus is 32 bits wide and each instruction is 64 bits long, then the processor must access the memory module twice during each instruction cycle. The address lines are used to designate the source or destination of the data on the data bus. For example, if the processor wishes to read a word (8, 16, or 32 bits) of data from memory, it puts the address of the desired word on the address lines. Clearly, the width of the address bus determines the maximum possible mem- ory capacity of the system. Furthermore, the address lines are generally also used to address I/O ports. Typically, the higher-order bits are used to select a particular module on the bus, and the lower-order bits select a memory location or I/O port within the module. For example, on an 8-bit address bus, address 01111111 and below might reference locations in a memory module (module 0) with 128 words of memory, and address 10000000 and above refer to devices attached to an I/O module (module 1). The control lines are used to control the access to and the use of the data and address lines. Because the data and address lines are shared by all components, there must be a means of controlling their use. Control signals transmit both com- mand and timing information among system modules. Timing signals indicate the validity of data and address information. Command signals specify operations to be performed. Typical control lines include Memory write: Causes data on the bus to be written into the addressed location Memory read: Causes data from the addressed location to be placed on the bus I/O write: Causes data on the bus to be output to the addressed I/O port I/O read: Causes data from the addressed I/O port to be placed on the bus Transfer ACK: Indicates that data have been accepted from or placed on the bus Bus request: Indicates that a module needs to gain control of the bus Bus grant: Indicates that a requesting module has been granted control of the bus Interrupt request: Indicates that an interrupt is pending Interrupt ACK: Acknowledges that the pending interrupt has been recognized Clock: Is used to synchronize operations Reset: Initializes all modules The operation of the bus is as follows. If one module wishes to send data to an- other, it must do two things: (1) obtain the use of the bus, and (2) transfer data via the bus. If one module wishes to request data from another module, it must (1) obtain the use of the bus, and (2) transfer a request to the other module over the appropriate control and address lines. It must then wait for that second module to send the data. Physically, the system bus is actually a number of parallel electrical con- ductors. In the classic bus arrangement, these conductors are metal lines etched in a card or board (printed circuit board). The bus extends across all of the sys- tem components, each of which taps into some or all of the bus lines. The classic physical arrangement is depicted in Figure 3.17. In this example, the bus consists  3.4 / BUS INTERCONNECTION 87 Bus Boards CPU Memory I/O Figure 3.17 Typical Physical Realization of a Bus Architecture of two vertical columns of conductors. At regular intervals along the columns, there are attachment points in the form of slots that extend out horizontally to support a printed circuit board. Each of the major system components occupies one or more boards and plugs into the bus at these slots. The entire arrangement is housed in a chassis. This scheme can still be used for some of the buses associ- ated with a computer system. However, modern systems tend to have all of the major components on the same board with more elements on the same chip as the processor. Thus, an on-chip bus may connect the processor and cache mem- ory, whereas an on-board bus may connect the processor to main memory and other components. This arrangement is most convenient. A small computer system may be ac- quired and then expanded later (more memory, more I/O) by adding more boards. If a component on a board fails, that board can easily be removed and replaced. Multiple-Bus Hierarchies If a great number of devices are connected to the bus, performance will suffer. There are two main causes: 1. In general, the more devices attached to the bus, the greater the bus length and hence the greater the propagation delay. This delay determines the time it takes for devices to coordinate the use of the bus. When control of the bus passes from one device to another frequently, these propagation delays can noticeably affect performance. 88 CHAPTER 3 / A TOP-LEVEL VIEW OF COMPUTER FUNCTION 2. The bus may become a bottleneck as the aggregate data transfer demand approaches the capacity of the bus. This problem can be countered to some extent by increasing the data rate that the bus can carry and by using wider buses (e.g., increasing the data bus from 32 to 64 bits). However, because the data rates generated by attached devices (e.g., graphics and video controllers, network interfaces) are growing rapidly, this is a race that a single bus is ulti- mately destined to lose. Accordingly, most computer systems use multiple buses, generally laid out in a hierarchy. A typical traditional structure is shown in Figure 3.18a. There is a local bus that connects the processor to a cache memory and that may support one or more local devices. The cache memory controller connects the cache not only to this local bus, but to a system bus to which are attached all of the main memory modules. As will be discussed in Chapter 4, the use of a cache structure insulates the processor from a requirement to access main memory frequently. Hence, main memory can be moved off of the local bus onto a system bus. In this way, I/O trans- fers to and from the main memory across the system bus do not interfere with the processor’s activity. It is possible to connect I/O controllers directly onto the system bus. A more efficient solution is to make use of one or more expansion buses for this purpose. An expansion bus interface buffers data transfers between the system bus and the I/O controllers on the expansion bus. This arrangement allows the system to support a wide variety of I/O devices and at the same time insulate memory-to-processor traf- fic from I/O traffic. Figure 3.18a shows some typical examples of I/O devices that might be attached to the expansion bus. Network connections include local area networks (LANs) such as a 10-Mbps Ethernet and connections to wide area networks (WANs) such as a packet-switching network. SCSI (small computer system interface) is itself a type of bus used to support local disk drives and other peripherals. A serial port could be used to support a printer or scanner. This traditional bus architecture is reasonably efficient but begins to break down as higher and higher performance is seen in the I/O devices. In response to these growing demands, a common approach taken by industry is to build a high- speed bus that is closely integrated with the rest of the system, requiring only a bridge between the processor’s bus and the high-speed bus. This arrangement is sometimes known as a mezzanine architecture. Figure 3.18b shows a typical realization of this approach. Again, there is a local bus that connects the processor to a cache controller, which is in turn connected to a system bus that supports main memory. The cache controller is integrated into a bridge, or buffering device, that connects to the high-speed bus. This bus supports connections to high-speed LANs, such as Fast Ethernet at 100 Mbps, video and graphics workstation controllers, as well as interface controllers to local peripheral buses, including SCSI and FireWire. The latter is a high-speed bus arrangement specifically designed to support high-capacity I/O devices. Lower-speed devices are still supported off an expansion bus, with an interface buffering traffic between the expansion bus and the high-speed bus. The advantage of this arrangement is that the high-speed bus brings high- demand devices into closer integration with the processor and at the same time is  3.4 / BUS INTERCONNECTION 89 Local bus Processor Cache Local I/O Main controller memory System bus Network Expansion bus interface Serial SCSI Modem Expansion bus (a) Traditional bus architecture Main memory Local bus Cache / Processor bridge System bus SCSI FireWire Graphic Video LAN High-speed bus FAX Expansion bus interface Serial Modem Expansion bus (b) High-performance architecture Figure 3.18 Example Bus Configurations independent of the processor. Thus, differences in processor and high-speed bus speeds and signal line definitions are tolerated. Changes in processor architecture do not affect the high-speed bus, and vice versa. Elements of Bus Design Although a variety of different bus implementations exist, there are a few basic pa- rameters or design elements that serve to classify and differentiate buses. Table 3.2 lists key elements. 90 CHAPTER 3 / A TOP-LEVEL VIEW OF COMPUTER FUNCTION Table 3.2 Elements of Bus Design Type Bus Width Dedicated Address Multiplexed Data Method of Arbitration Data Transfer Type Centralized Read Distributed Write Timing Read-modify-write Synchronous Read-after-write Asynchronous Block BUS TYPES Bus lines can be separated into two generic types: dedicated and multi- plexed. A dedicated bus line is permanently assigned either to one function or to a physical subset of computer components. An example of functional dedication is the use of separate dedicated address and data lines, which is common on many buses. However, it is not essential. For ex- ample, address and data information may be transmitted over the same set of lines using an Address Valid control line. At the beginning of a data transfer, the address is placed on the bus and the Address Valid line is activated. At this point, each mod- ule has a specified period of time to copy the address and determine if it is the ad- dressed module. The address is then removed from the bus, and the same bus connections are used for the subsequent read or write data transfer. This method of using the same lines for multiple purposes is known as time multiplexing. The advantage of time multiplexing is the use of fewer lines, which saves space and, usually, cost. The disadvantage is that more complex circuitry is needed within each module. Also, there is a potential reduction in performance because certain events that share the same lines cannot take place in parallel. Physical dedication refers to the use of multiple buses, each of which connects only a subset of modules. A typical example is the use of an I/O bus to interconnect all I/O modules; this bus is then connected to the main bus through some type of I/O adapter module. The potential advantage of physical dedication is high throughput, because there is less bus contention. A disadvantage is the increased size and cost of the system. METHOD OF ARBITRATION In all but the simplest systems, more than one module may need control of the bus. For example, an I/O module may need to read or write directly to memory, without sending the data to the processor. Because only one unit at a time can successfully transmit over the bus, some method of arbitration is needed. The various methods can be roughly classified as being either centralized or distributed. In a centralized scheme, a single hardware device, referred to as a bus controller or arbiter, is responsible for allocating time on the bus. The device may be a separate module or part of the processor. In a distributed scheme, there is no cen- tral controller. Rather, each module contains access control logic and the modules act together to share the bus. With both methods of arbitration, the purpose is to designate one device, either the processor or an I/O module, as master. The master  3.4 / BUS INTERCONNECTION 91 may then initiate a data transfer (e.g., read or write) with some other device, which acts as slave for this particular exchange. TIMING Timing refers to the way in which events are coordinated on the bus. Buses use either synchronous timing or asynchronous timing. With synchronous timing, the occurrence of events on the bus is determined by a clock. The bus includes a clock line upon which a clock transmits a regular se- quence of alternating 1s and 0s of equal duration. A single 1–0 transmission is re- ferred to as a clock cycle or bus cycle and defines a time slot. All other devices on the bus can read the clock line, and all events start at the beginning of a clock cycle. Figure 3.19 shows a typical, but simplified, timing diagram for synchronous read and write operations (see Appendix 3A for a description of timing dia- grams). Other bus signals may change at the leading edge of the clock signal (with a slight reaction delay). Most events occupy a single clock cycle. In this simple ex- ample, the processor places a memory address on the address lines during the first T1 T2 T3 Clock Status Status signals lines Address Stableaddress Stable address lines Address enable Data Valid data in lines Read cycle Read Data Valid data out Write lines cycle Write Figure 3.19 Timing of Synchronous Bus Operations 92 CHAPTER 3 / A TOP-LEVEL VIEW OF COMPUTER FUNCTION clock cycle and may assert various status lines. Once the address lines have stabi- lized, the processor issues an address enable signal. For a read operation, the processor issues a read command at the start of the second cycle. A memory mod- ule recognizes the address and, after a delay of one cycle, places the data on the data lines. The processor reads the data from the data lines and drops the read sig- nal. For a write operation, the processor puts the data on the data lines at the start of the second cycle, and issues a write command after the data lines have stabi- lized. The memory module copies the information from the data lines during the third clock cycle. With asynchronous timing, the occurrence of one event on a bus follows and depends on the occurrence of a previous event. In the simple read example of Figure 3.20a, the processor places address and status signals on the bus. After Status Status signals lines Address lines Stable address Read Data lines Valid data Acknowledge (a) System bus read cycle Status Status signals lines Address lines Stable address Data lines Valid data Write Acknowledge (b) System bus write cycle Figure 3.20 Timing of Asynchronous Bus Operations  3.4 / BUS INTERCONNECTION 93 pausing for these signals to stabilize, it issues a read command, indicating the pres- ence of valid address and control signals. The appropriate memory decodes the ad- dress and responds by placing the data on the data line. Once the data lines have stabilized, the memory module asserts the acknowledged line to signal the proces- sor that the data are available. Once the master has read the data from the data lines, it deasserts the read signal. This causes the memory module to drop the data and acknowledge lines. Finally, once the acknowledge line is dropped, the master removes the address information. Figure 3.20b shows a simple asynchronous write operation. In this case, the master places the data on the data line at the same time that is puts signals on the status and address lines. The memory module responds to the write command by copying the data from the data lines and then asserting the acknowledge line. The master then drops the write signal and the memory module drops the acknowl- edge signal. Synchronous timing is simpler to implement and test. However, it is less flexi- ble than asynchronous timing. Because all devices on a synchronous bus are tied to a fixed clock rate, the system cannot take advantage of advances in device perfor- mance. With asynchronous timing, a mixture of slow and fast devices, using older and newer technology, can share a bus. BUS WIDTH We have already addressed the concept of bus width. The width of the data bus has an impact on system performance: The wider the data bus, the greater the number of bits transferred at one time. The width of the address bus has an im- pact on system capacity: the wider the address bus, the greater the range of locations that can be referenced. DATA TRANSFER TYPE Finally, a bus supports various data transfer types, as illus- trated in Figure 3.21. All buses support both write (master to slave) and read (slave to master) transfers. In the case of a multiplexed address/data bus, the bus is first used for specifying the address and then for transferring the data. For a read opera- tion, there is typically a wait while the data are being fetched from the slave to be put on the bus. For either a read or a write, there may also be a delay if it is necessary to go through arbitration to gain control of the bus for the remainder of the opera- tion (i.e., seize the bus to request a read or write, then seize the bus again to perform a read or write). In the case of dedicated address and data buses, the address is put on the ad- dress bus and remains there while the data are put on the data bus. For a write oper- ation, the master puts the data onto the data bus as soon as the address has stabilized and the slave has had the opportunity to recognize its address. For a read operation, the slave puts the data onto the data bus as soon as it has recognized its address and has fetched the data. There are also several combination operations that some buses allow. A read–modify–write operation is simply a read followed immediately by a write to the same address. The address is only broadcast once at the beginning of the operation. The whole operation is typically indivisible to prevent any access to the data element by other potential bus masters. The principal purpose of this 94 CHAPTER 3 / A TOP-LEVEL VIEW OF COMPUTER FUNCTION Time Time Address Data Address Data and address (1st cycle) (2nd cycle) sent by master Write (multiplexed) operation in same cycle over Data separate bus lines. Write (non-multiplexed) operation Access Address Data time Read (multiplexed) operation Time Address Data Data Address Data read write Read-modify-write operation Read (non-multiplexed) operation Data Data Address write read Read-after-write operation Address Data Data Data Block data transfer Figure 3.21 Bus Data Transfer Types capability is to protect shared memory resources in a multiprogramming system (see Chapter 8). Read-after-write is an indivisible operation consisting of a write followed im- mediately by a read from the same address. The read operation may be performed for checking purposes. Some bus systems also support a block data transfer. In this case, one address cycle is followed by n data cycles. The first data item is transferred to or from the specified address; the remaining data items are transferred to or from subsequent addresses.  3.5 / PCI 95 3.5 PCI The peripheral component interconnect (PCI) is a popular high-bandwidth, processor-independent bus that can function as a mezzanine or peripheral bus. Compared with other common bus specifications, PCI delivers better system per- formance for high-speed I/O subsystems (e.g., graphic display adapters, network interface controllers, disk controllers, and so on). The current standard allows the use of up to 64 data lines at 66 MHz, for a raw transfer rate of 528 MByte/s, or 4.224 Gbps. But it is not just a high speed that makes PCI attractive. PCI is specif- ically designed to meet economically the I/O requirements of modern systems; it requires very few chips to implement and supports other buses attached to the PCI bus. Intel began work on PCI in 1990 for its Pentium-based systems. Intel soon re- leased all the patents to the public domain and promoted the creation of an industry association, the PCI Special Interest Group (SIG), to develop further and maintain the compatibility of the PCI specifications. The result is that PCI has been widely adopted and is finding increasing use in personal computer, workstation, and server systems. Because the specification is in the public domain and is supported by a broad cross section of the microprocessor and peripheral industry, PCI products built by different vendors are compatible. PCI is designed to support a variety of microprocessor-based configurations, including both single- and multiple-processor systems. Accordingly, it provides a general-purpose set of functions. It makes use of synchronous timing and a central- ized arbitration scheme. Figure 3.22a shows a typical use of PCI in a single-processor system. A com- bined DRAM controller and bridge to the PCI bus provides tight coupling with the processor and the ability to deliver data at high speeds. The bridge acts as a data buffer so that the speed of the PCI bus may differ from that of the processor’s I/O capability. In a multiprocessor system (Figure 3.22b), one or more PCI configura- tions may be connected by bridges to the processor’s system bus. The system bus supports only the processor/cache units, main memory, and the PCI bridges. Again, the use of bridges keeps the PCI independent of the processor speed yet provides the ability to receive and deliver data rapidly. Bus Structure PCI may be configured as a 32- or 64-bit bus. Table 3.3 defines the 49 mandatory sig- nal lines for PCI. These are divided into the following functional groups: System pins: Include the clock and reset pins. Address and data pins: Include 32 lines that are time multiplexed for ad- dresses and data. The other lines in this group are used to interpret and vali- date the signal lines that carry the addresses and data. Interface control pins: Control the timing of transactions and provide coordi- nation among initiators and targets. 96 CHAPTER 3 / A TOP-LEVEL VIEW OF COMPUTER FUNCTION Processor Cache Motion Bridge/ Audio video memory DRAM controller PCI Bus LAN SCSI Expansion bus bridge Graphics Base I/O devices Expansion bus (a) Typical desktop system Processor/ Processor/ Memory cache cache controller DRAM System bus Host bridge Host bridge PCI Bus PCI Bus Expansion Expansion SCSI SCSI LAN LAN bus bridge bus bridge PCI to PCI bridge (b) Typical server system Figure 3.22 Example PCI Configurations Arbitration pins: Unlike the other PCI signal lines, these are not shared lines. Rather, each PCI master has its own pair of arbitration lines that connect it di- rectly to the PCI bus arbiter. Error reporting pins: Used to report parity and other errors.  3.5 / PCI 97 Table 3.3 Mandatory PCI Signal Lines Designation Type Description System Pins CLK in Provides timing for all transactions and is sampled by all inputs on the rising edge. Clock rates up to 33 MHz are supported. RST# in Forces all PCI-specific registers, sequencers, and signals to an initialized state. Address and Data Pins AD[31::0] t/s Multiplexed lines used for address and data C/BE[3::0]# t/s Multiplexed bus command and byte enable signals. During the data phase, the lines indicate which of the four byte lanes carry meaningful data. PAR t/s Provides even parity across AD and C/BE lines one clock cycle later. The master drives PAR for address and write data phases; the target drive PAR for read data phases. Interface Control Pins FRAME# s/t/s Driven by current master to indicate the start and duration of a transaction. It is as- serted at the start and deasserted when the initiator is ready to begin the final data phase. IRDY# s/t/s Initiator Ready. Driven by current bus master (initiator of transaction). During a read, indicates that the master is prepared to accept data; during a write, indicates that valid data are present on AD. TRDY# s/t/s Target Ready. Driven by the target (selected device). During a read, indicates that valid data are present on AD; during a write, indicates that target is ready to accept data. STOP# s/t/s Indicates that current target wishes the initiator to stop the current transaction. IDSEL in Initialization Device Select. Used as a chip select during configuration read and write transactions. DEVSEL# in Device Select. Asserted by target when it has recognized its address. Indicates to cur- rent initiator whether any device has been selected. Arbitration Pins REQ# t/s Indicates to the arbiter that this device requires use of the bus. This is a device- specific point-to-point line. GNT# t/s Indicates to the device that the arbiter has granted bus access. This is a device- specific point-to-point line. Error Reporting Pins PERR# s/t/s Parity Error. Indicates a data parity error is detected by a target during a write data phase or by an initiator during a read data phase. SERR# o/d System Error. May be pulsed by any device to report address parity errors and critical errors other than parity. In addition, the PCI specification defines 51 optional signal lines (Table 3.4), divided into the following functional groups: Interrupt pins: These are provided for PCI devices that must generate re- quests for service. As with the arbitration pins, these are not shared lines. Rather, each PCI device has its own interrupt line or lines to an interrupt controller. 98 CHAPTER 3 / A TOP-LEVEL VIEW OF COMPUTER FUNCTION Table 3.4 Optional PCI Signal Lines Designation Type Description Interrupt Pins INTA# o/d Used to request an interrupt. INTB# o/d Used to request an interrupt; only has meaning on a multifunction device. INTC# o/d Used to request an interrupt; only has meaning on a multifunction device. INTD# o/d Used to request an interrupt; only has meaning on a multifunction device. Cache Support Pins SBO# in/out Snoop Backoff. Indicates a hit to a modified line. SDONE in/out Snoop Done. Indicates the status of the snoop for the current access. Asserted when snoop has been completed. 64-Bit Bus Extension Pins AD[63::32] t/s Multiplexed lines used for address and data to extend bus to 64 bits. C/BE[7::4]# t/s Multiplexed bus command and byte enable signals. During the address phase, the lines provide additional bus commands. During the data phase, the lines indicate which of the four extended byte lanes carry meaningful data. REQ64# s/t/s Used to request 64-bit transfer. ACK64# s/t/s Indicates target is willing to perform 64-bit transfer. PAR64 t/s Provides even parity across extended AD and C/BE lines one clock cycle later. JTAG/Boundary Scan Pins TCK in Test clock. Used to clock state information and test data into and out of the device during boundary scan. TDI in Test input. Used to serially shift test data and instructions into the device. TDO out Test output. Used to serially shift test data and instructions out of the device. TMS in Test mode Select. Used to control state of test access port controller. TRST# in Test reset. Used to initialize test access port controller. in Input-only signal out Output-only signal t/s Bidirectional, tri-state, I/O signal s/t/s Sustained tri-state signal driven by only one owner at a time o/d Open drain: allows multiple devices to share as a wire-OR # Signal’s active state occurs at low voltage Cache support pins: These pins are needed to support a memory on PCI that can be cached in the processor or another device. These pins support snoopy cache protocols (see Chapter 18 for a discussion of such protocols). 64-bit bus extension pins: Include 32 lines that are time multiplexed for ad- dresses and data and that are combined with the mandatory address/data lines to form a 64-bit address/data bus. Other lines in this group are used to interpret and validate the signal lines that carry the addresses and data. Finally, there are two lines that enable two PCI devices to agree to the use of the 64-bit capability. JTAG/boundary scan pins: These signal lines support testing procedures de- fined in IEEE Standard 1149.1.  3.5 / PCI 99 PCI Commands Bus activity occurs in the form of transactions between an initiator, or master, and a target. When a bus master acquires control of the bus, it determines the type of transaction that will occur next. During the address phase of the transaction, the C/BE lines are used to signal the transaction type. The commands are as follows: Interrupt Acknowledge Special Cycle I/O Read I/O Write Memory Read Memory Read Line Memory Read Multiple Memory Write Memory Write and Invalidate Configuration Read Configuration Write Dual address Cycle Interrupt Acknowledge is a read command intended for the device that func- tions as an interrupt controller on the PCI bus. The address lines are not used during the address phase, and the byte enable lines indicate the size of the interrupt identi- fier to be returned. The Special Cycle command is used by the initiator to broadcast a message to one or more targets. The I/O Read and Write commands are used to transfer data between the initia- tor and an I/O controller. Each I/O device has its own address space, and the address lines are used to indicate a particular device and to specify the data to be transferred to or from that device. The concept of I/O addresses is explored in Chapter 7. The memory read and write commands are used to specify the transfer of a burst of data, occupying one or more clock cycles. The interpretation of these com- mands depends on whether or not the memory controller on the PCI bus supports the PCI protocol for transfers between memory and cache. If so, the transfer of data to and from the memory is typically in terms of cache lines, or blocks.3 The three memory read commands have the uses outlined in Table 3.5. The Memory Write command is used to transfer data in one or more data cycles to memory. The Mem- ory Write and Invalidate command transfers data in one or more cycles to memory. In addition, it guarantees that at least one cache line is written. This command sup- ports the cache function of writing back a line to memory. The two configuration commands enable a master to read and update configu- ration parameters in a device connected to the PCI. Each PCI device may include 3 The fundamental principles of cache memory are described in Chapter 4; bus-based cache protocols are described in Chapter 17. 100 CHAPTER 3 / A TOP-LEVEL VIEW OF COMPUTER FUNCTION Table 3.5 Interpretation of PCI Read Commands Read Command Type For Cachable Memory For Noncachable Memory Memory Read Bursting one-half or less of a Bursting 2 data transfer cycles cache line or less Memory Read Line Bursting more than one-half a Bursting 3 to 12 data transfers cache line to three cache lines Memory Read Multiple Bursting more than three cache Bursting more than 12 data lines transfers up to 256 internal registers that are used during system initialization to configure that device. The Dual Address Cycle command is used by an initiator to indicate that it is using 64-bit addressing. Data Transfers Every data transfer on the PCI bus is a single transaction consisting of one address phase and one or more data phases. In this discussion, we illustrate a typical read operation; a write operation proceeds similarly. Figure 3.23 shows the timing of the read transaction. All events are synchro- nized to the falling transitions of the clock, which occur in the middle of each clock cycle. Bus devices sample the bus lines on the rising edge at the beginning of a bus cycle. The following are the significant events, labeled on the diagram: a. Once a bus master has gained control of the bus, it may begin the transaction by asserting FRAME. This line remains asserted until the initiator is ready to complete the last data phase. The initiator also puts the start address on the ad- dress bus, and the read command on the C/BE lines. b. At the start of clock 2, the target device will recognize its address on the AD lines. c. The initiator ceases driving the AD bus. A turnaround cycle (indicated by the two circular arrows) is required on all signal lines that may be driven by more than one device, so that the dropping of the address signal will prepare the bus for use by the target device. The initiator changes the information on the C/BE lines to designate which AD lines are to be used for transfer for the currently addressed data (from 1 to 4 bytes). The initiator also asserts IRDY to indicate that it is ready for the first data item. d. The selected target asserts DEVSEL to indicate that it has recognized its ad- dress and will respond. It places the requested data on the AD lines and as- serts TRDY to indicate that valid data are present on the bus. e. The initiator reads the data at the beginning of clock 4 and changes the byte enable lines as needed in preparation for the next read. f. In this example, the target needs some time to prepare the second block of data for transmission. Therefore, it