Unit 02 Software Testing Levels (1).pptx

SWE415 OR SENG-364 SOFTWARE TESTING AND QUALITY ASSURANCE. Lecture 2 Software Testing Levels Software is like entropy. It is difficult to grasp, weighs nothing, and obeys the second law of thermodynamics, i.e., it always increases. — Norman Ralph Augustine Testing Levels Testing is performed...

SWE415 OR SENG-364 SOFTWARE TESTING AND QUALITY ASSURANCE. Lecture 2 Software Testing Levels Software is like entropy. It is difficult to grasp, weighs nothing, and obeys the second law of thermodynamics, i.e., it always increases. — Norman Ralph Augustine Testing Levels Testing is performed at different levels involving the complete system or parts of it throughout the life cycle of a software product. A software system goes through four stages of testing before it is actually deployed. Unit Testing. Integration Testing. System Testing. Operational Acceptance Testing. Testing Levels The first three levels of testing are performed by a number of different stakeholders in the development organization, where as acceptance testing is performed by the customers. Software Development Life Cycle (SDLC) Software Development Life Cycle (SDLC) Testing Levels Unit Testing Unit testing refers to testing program units in isolation that is, in a stand-alone manner A unit is the smallest testable part of software. A program unit is a piece of code, such as Functions, Procedures, Methods, or Class. Unit Testing is performed by using the White Box Testing method. Unit Testing is the first level of testing and is performed prior to Integration Testing. Unit Testing There are two reasons for testing a unit in a stand-alone manner: Errors found during testing can be attributed to a specific unit so that it can be easily fixed. Unit testing removes dependencies on other program units. During unit testing it is desirable to verify that each distinct execution of a program unit produces the expected result. A distinct execution refers to a distinct path in the unit. A programmer will need to verify whether or not a code works correctly by performing unit-level testing. Unit Testing A programmer needs to test a unit as follows: Execute every line of code. This is desirable because the programmer needs to know what happens when a line of code is executed. In the absence of such basic observations, surprises at a later stage can be expensive. Execute every predicate in the unit to evaluate them to true and false separately. Observe that the unit performs its intended function and ensure that it contains no known errors. Unit Testing Who perform Unit Testing? Unit testing is performed by the programmer (Developer) who writes the program unit because the programmer is intimately familiar with the internal details of the unit. The defects found during unit testing are internal to the software development group and are not reported up the personnel hierarchy to be counted in quality measurement metrics. Tools For Unit Testing 1. Code Auditor: This tool is used to check the quality of software to ensure that it meets some minimum coding standards. 2. Bound Checker: This tool can check for accidental writes into the instruction areas of memory or to any other memory location outside the data storage area of the application. 3. Documenters: These tools read source code and automatically generate descriptions and caller/callee tree diagram or data model from the source code. Tools For Unit Testing 4. Memory Leak Detectors: These tools test the allocation of memory to an application which requests for memory, but fails to de-allocate. 5. Static Code (Path) Analyzer: These tools identify paths to test, based on the structure of the code such as McCabe’s cyclomatic complexity measure. 6. Software Inspection Support: Tools can help schedule group inspections. These can also provide status of items reviewed and follow-up actions and distribute the reports of problem resolution. They can be integrated with other tools, such as static code analyzers. Tools For Unit Testing 7. Test Coverage Analyzer: These tools measure internal test coverage, often expressed in terms of the control structure of the test object, and report the coverage metric. Coverage analyzers track and report what paths were exercised during dynamic unit testing. 8. See pages 77 - 80 Banking System Scenario You work as a Software Tester. Your company is hired by a bank to develop an online banking system. 1. Requirement Analysis 1. Login on valid Credential 2. View Current Balance 3. Deposit 4. Withdraw 5. Transfer Banking System Scenario 2. Functional Specification Banking System Scenario 3. High Level Design Banking System Scenario 4. Detail Design Unit Testing How to test this component JUnit The JUnit is a unit testing framework for the Java programming language designed by Kent Beck and Erich Gamma. http://junit.org/junit4/ Integration Testing Integration testing is any type of software testing that seeks to verify the interfaces between components against a software design. The purpose of integration testing is to verify the functional, performance, and reliability between the modules that are integrated. Integration Testing The objective of system integration is to build a “working” version of the system by: Putting the modules together in an incremental manner. Ensuring that the additional modules work as expected without disturbing the functionalities of the modules already put together. System Integration Testing Advantages Integration testing is said to be complete when the system is fully integrated together, All the test cases have been executed, All the severe and moderate defects found have been fixed, and the system is retested. The major advantages of conducting system integration testing are as follows: Defects are detected early. It is easier to fix defects detected earlier. We get earlier feedback on the health and acceptability of the individual modules and on the overall system. Scheduling of defect fixes is flexible, and it can overlap with development. Who perform integration testing? System integration testing is performed by the System Integration Group. Software Developers Integration Test Engineers The integration test engineers need to know the details of the software modules. This means that the team of engineers who built the modules needs to be involved in system integration. The system architects should be involved in the integration testing of complex software systems because of the fact that they have the bigger picture of the system. Test Team We cannot solve our problems with the same thinking we used when we created them. Albert Einstein Professional Tester System Designer Analyst Test Team Configuration User Management Specialist Programmer Different Types Of Interfaces An interface between two modules allows one module to access the service provided by the other. It implements a mechanism for passing control and data between modules. Three common paradigms for interfacing modules are as follows: Procedure Call Interface: A procedure in one module calls a procedure in another module. The caller passes on control to the called module. The caller can pass data to the called procedure, and the called procedure can pass data to the caller while returning control back to the caller. Different Types Of Interfaces Shared Memory Interface: A block of memory is shared between two modules. The memory block may be allocated by one of the two modules or a third module. Data are written into the memory block by one module and are read from the block by the other. Message Passing Interface: One module prepares a message by initializing the fields of a data structure and sending the message to another module. This form of module interaction is common in client–server-based systems and web-based systems. Interface Errors The question is: If all the unit-tested modules work individually, why can these modules not work when put together? The problem arises when we “put them together” because of rampant interface errors. Perry and Evangelist reported in 1987 that interface errors accounted for up to a quarter of all errors in the systems they examined. They found that of all errors that required a fix within one module, more than half were caused by interface errors. See Chapter 7 pages 160 – 162 System Integration Testing Techniques Integration testing need not wait until all the modules of a system are coded and unit tested. Instead, it can begin as soon as the relevant modules are available. Some common approaches to performing system integration are as follows: Incremental Top down Bottom up Sandwich Big bang Incremental Approach Integration testing is conducted in an incremental manner as a series of test cycles as suggested by Deutsch. In each test cycle, a few more modules are integrated with an existing and tested build to generate a larger build. The idea is to complete one cycle of testing, let the developers fix all the errors found, and continue the next cycle of testing. The complete system is built incrementally, cycle by cycle, until the whole system is operational and ready for System-level testing. Top-Down Approach Systems with hierarchical structures easily lend themselves to top-down and bottom-up approaches to integration. In a hierarchical system, there is a first, top-level module which is decomposed into a few second-level modules. Some of the second-level modules may be further decomposed into third-level modules, and so on. A terminal module is one that is no more decomposed. See Chapter 7 pages 167 – 171. Top-Down Approach Example Bottom-Up Approach System Integration begins with the integration of lowest level modules. A module is said to be at the lowest level if it does not invoke another module. To integrate a set of lower-level modules in this approach, we need to construct a test driver module that invokes the modules to be integrated. Bottom Up Approach Example Sandwich and Big Bang Approach Sandwich Approach, a system is integrated by using a mix of the top-down and bottom-up approaches. Big-Bang Approach, first all the modules are individually tested. Next, all those modules are put together to construct the entire system which is tested as a whole. For large systems, this approach is not recommended for the following reasons: In a system with a large number of modules, there may be many interface defects. It is difficult to determine whether or not the cause of a failure is due to interface errors in a large and complex system. In large systems, the presence of a large number of interface errors is not an unlikely scenario in software development. Thus, it is not cost effective to be optimistic by putting the modules together and hoping it will work. Test Plan For System Integration Integration Testing Example “I am the owner of an advertising company and I post ads on different websites. At the end of the month, I want to see how many people saw my ads and how many people clicked on my ads. I need a report for my ads displayed and I charge accordingly to my clients.” GenNext software developed this product for me and below was the architecture: Integration Testing Example Integration Testing Example UI – User Interface module, which is visible to the end user, where all the inputs are given. BL – Is the Business Logic module, which has all the all the calculations and business specific methods. VAL – Is the Validation module, which has all the validations of the correctness of the input. CNT – Is the content module which has all the static contents, specific to the inputs entered by the user. These contents are displayed in the reports. EN – Is the Engine module, this module reads all the data that comes from BL, VAL and CNT module and extracts the SQL query and triggers it to the database. Scheduler – Is a module which schedules all the reports based on the user selection (monthly, quarterly, semiannually & annually) DB – Is the Database. Integration Testing Example To perform Integration Testing, you have to answer the following questions; 1. How the BL, VAL and the CNT module will read and interpret ate the data entered in the UI module? 2. Is BL, VAL and CNT module receiving the correct data from UI? 3. In which format the data from BL, VAL and CNT is transferred to the EN module? 4. How will the EQ read the data and extract the query? 5. Is the query extracted correctly? 6. Is the Scheduler getting the correct data for reports? 7. Is the result set received by the EN, from the database is correct and as expected? 8. Is EN able to send the response back to the BL, VAL and CNT module? 9. Is UI module able to read the data and display it appropriately to the interface? Integration Testing Example In the real world, the communication of data is done in an XML format. So whatever data the user enters in the UI, it gets converted into an XML format. Well, testing whether the information/data is flowing correctly or not will be your integration testing, which in this case would be validating the XML files. Are the XML files generated correctly? Do they have the correct data? Are the data is being transferred correctly from one module to another? All these things will be tested as part of Integration testing. System Testing System testing is a critical phase in a software development process because of the need to meet a tight schedule close to delivery date, to discover most of the faults, and to verify that fixes are working and have not resulted in new faults. The objective of system testing, is to establish whether an implementation conforms to the requirements specified by the customers. It takes much effort to guarantee that customer requirements have been met and the system is acceptable. System Testing Activities System testing comprises a number of distinct activities: Creating a test plan Designing a test suite Preparing test environments Executing the tests by following a clear strategy Monitoring the process of test execution System Testing Types We will explain these type in detail later System Testing Types advantages 1. Test engineers can accurately focus on different aspects of a system, one at a time, while evaluating its quality. 2. Test engineers can prioritize their tasks based on test categories. For example, it is more meaningful and useful to identify the limitations of a system only after ensuring that the system performs all basic functions to the test an engineer’s satisfactions. Therefore, stress tests, which thrive to identify the limitations of a system, are executed after functionality tests. System Testing Types advantages 3. Planning the system testing phase based on test categorization lets a test engineer obtain a well-balanced test suite. Practical limitations make it difficult to be exhaustive, and economic considerations may restrict the testing process from continuing any further. However, it is important to design a balanced test suite, rather than an unbalanced one with many test cases in one category and no tests in another. Operational Acceptance Testing The objective of acceptance testing is to measure the quality of the product, rather than searching for the defects. Acceptance testing is performed by the customers based on their expectations from the product. For example, The service provider of a cellular phone network is a customer of the software systems running the phone network, whereas the general public forms the user base by subscribing to the phone services. It is not uncommon for someone to have a dual role as a customer and a user. Acceptance Testing Categories There are two categories of acceptance testing: User acceptance testing is conducted by the customer to ensure that system satisfies the contractual acceptance criteria before being signed off as meeting user needs. Actual planning and execution of the acceptance tests do not have to be undertaken directly by the customer. Business acceptance testing is undertaken within the development organization of the supplier to ensure that the system will eventually pass the UAT. Test Case A test case contains: A sequence of Steps describing actions to be performed, Test data to be used An expected response for each action performed. Test Cases are written based on Business and Functional/Technical requirements, use cases and Technical design documents. The level of details specified in test cases will vary depending on Organizations, Projects and also on the Test Case Template used OR on the Test Management tool being used in the project. Test Case Construction of Test Cases also helps in: Finding issues or gaps in the requirements Technical design itself. As Test Case construction activity would make tester to think through different possible Positive and Negative scenarios. It is test cases against which tester will verify the application is working as expected. Number of test cases to be created depends on the size, complexity and type of testing being performed Test Case Examples Test case for ATM TC 1 :- successful card insertion. TC 2 :- unsuccessful operation due to wrong angle card insertion. TC 3:- unsuccessful operation due to invalid account card. TC 4:- successful entry of pin number. TC 5:- unsuccessful operation due to wrong pin number entered 3 times. TC 6:- successful selection of language. TC 7:- successful selection of account type. TC 8:- unsuccessful operation due to wrong account type selected w/r to that inserted card. TC 9:- successful selection of withdrawal option. TC 10:- successful selection of amount. TC 11:- unsuccessful operation due to wrong denominations. TC 12:- successful withdrawal operation. TC 13:- unsuccessful withdrawal operation due to amount greater than possible balance. TC 14:- unsuccessful due to lack of amount in ATM. TC 15:- un due to amount greater than the day limit. TC 16:- un due to server down. TC 17:- un due to click cancel after insert card. TC 18:- un due to click cancel after insert card and pin no. TC 19:- un due to click cancel after language selection, account type selection, withdrawal selection, enter amount Test Case Examples Test cases for a log-in web page Testing without entering any username and password Test it only with Username Test it only with password. User name with wrong password Password with wrong user name Right username and right password Cancel, after entering username and password. Enter long username and password that exceeds the set limit of characters. Try copy/paste in the password text box. After successful sign-out, try “Back” option from your browser. Check whether it gets you to the “signed-in” page. Source of Test Case Requirements Use cases Design documents Test Case Template Test Case #: Test Case Name: System: Subsystem: Designed by: Design Date: Executed by: Execution Date: Short Description: Pre-conditions: Step Action Expected System Response Pass/ Fail Comment 1 2 3 Post-conditions: Test Case Example Test Case #: TC123 Test Case Name: Change PIN System: ATM Subsystem: PIN Designed by: ABC Design Date: 28/11/2004 Executed by: Execution Date: Short Description: Test the ATM Change PIN service Pre-conditions: The user has a valid ATM card - The user has accessed the ATM by placing his ATM card in the machine The current PIN is 1234 The system displays the main menu Step Action Expected System Response Pass/ Fail Comment 1 Click the 'Change PIN' button The system displays a message asking the user to enter the new PIN 2 Enter '5555' The system displays a message asking the user to confirm (re-enter) the new PIN 3 Re-enter '5555' The system displays a message of successful operation The system asks the user if he wants to perform other operations 4 Click 'YES' button The system displays the main menu 5 Check post-condition 1 Post-conditions: 1. The new PIN '5555' is saved in the database Class Exercise. Write a test case for successful withdrawal operation in ATM system? Use the test case example in Slide 43. A Testing Class Exercise A program reads three integer values from a card. The three values are interpreted as representing the lengths of the sides of a triangle. The program prints a message that states whether the triangle is scalene, isosceles, or equilateral. On a sheet of paper, write a set of test cases (i.e., specific sets of data) that you feel would adequately test this program. Summary