Software Testing: Processes, Strategies & Techniques - PDF
Document Details
![NeatestStrength](https://quizgecko.com/images/avatars/avatar-7.webp)
Uploaded by NeatestStrength
Glasgow Caledonian University
Tags
Summary
This document covers software testing processes and strategies, including unit testing, integration testing, and system testing. It also examines debugging, various testing techniques like black box and white box testing, and the importance of test case design for accurate software validation. The content focuses on ensuring software quality and the identification/elimination of errors.
Full Transcript
SOFTWARE PROCESSES AND PRACTICE SOFTWARE TESTING 1 Introduction ⚫ Testing is a process through which a program is validated ⚫ Positive activity that seeks to show that the program meets its design specification ⚫ The purpose of testing is to fi...
SOFTWARE PROCESSES AND PRACTICE SOFTWARE TESTING 1 Introduction ⚫ Testing is a process through which a program is validated ⚫ Positive activity that seeks to show that the program meets its design specification ⚫ The purpose of testing is to find errors/bugs ⚫ Should be planned in advance and begun in the early stages of the development ⚫ Debugging is the process whereby errors are removed ⚫ It is a negative reactive activity which stems from testing but cannot be planned ahead of time. 2 The myth ⚫ “So goes the myth There are bugs, the myth says, because we are bad at what we do; and if we are bad at it, we should feel guilty’ about it. Therefore, testing and test design amount to an admission of failure, which instils a goodly dose of guilt. And the tedium of testing is just punishment for our errors.” ⚫ Quote from Boris Beizer (who goes on to refute this myth!) 3 Bugs are inevitable ⚫ Research has shown that even good programs will still have between 1 and 3 bugs for every hundred statements ⚫ Therefore programmers have to adjust their view about testing and debugging because it is an inevitable part of the process which should be encountered enthusiastically ⚫ After all, the art of debugging particularly a stubborn bug can prove more challenging than a simple coding exercise. 4 Debugging 5 Objectives of testing ⚫ To affirm the quality of the product ⚫ To find and eliminate any residual errors from previous stages ⚫ To demonstrate the presence of all specified functionality ⚫ To validate software as a solution for the original problem ⚫ To estimate the operational reliability of the system 6 Verification and validation ⚫ Verification ⚫ “Are we building the product right?” ⚫ Validation ⚫ “Are we building the right product?” 7 Stages of testing ⚫ Unit testing ⚫ Individual modules tested as standalone entities ⚫ A “unit” in OO may be one class, or a few related classes ⚫ Integration testing ⚫ Test sets of individual components making up subsystems ⚫ May function properly separately but not in combination, e.g. interface mismatches 8 Stages of testing ⚫ Validation testing ⚫ Whole system tested in a realistic setting ⚫ Alpha test - developer’s site by a customer, controlled environment ⚫ Beta test - one or more customer’s sites by end user developer generally not present customer records all problems and reports to developer ⚫ Acceptance test - Formal testing conducted to enable a user, customer, or other authorized entity to determine whether to accept a system or component 9 Stages of testing ⚫ System testing ⚫ This involves testing issues which concern the whole system and its relationship with other systems ⚫ Performance, security, configuration are all issues ⚫ Performance testing, often known as load testing, is particularly important in applications with multiple simultaneous users, for example network or web-based applications 10 Testing strategies ⚫ Testing begins at module level and works outward ⚫ Different testing techniques for each stage ⚫ Testing by developers and sometimes by independent test team ⚫ Debugging required at all stages 11 Unit testing issues ⚫ Need to devise test cases which give likelihood of finding errors ⚫ Test case design should use techniques which give as wide coverage as possible for exercising the software unit ⚫ Two key techniques ⚫ Black Box (Functional) testing ⚫ White Box (Structural) testing 12 Black Box testing ⚫ Software element under test is considered a “black box” ⚫ Internal operations are of no concern to the tester ⚫ Test cases are constructed from the functional specification of the unit ⚫ Tests that when inputs are accepted, the corresponding outputs are correctly produced 13 White Box testing ⚫ Fine procedural detail of the unit is used in construction of test cases ⚫ Actual logic paths through the code are tested ⚫ Exercise specific sets of conditions and loops ⚫ The status of the program can be examined 14 TDD – black or white? ⚫ Unit tests in TDD should involve mainly black box testing, as tests are written before code ⚫ Tests based on specification of functionality to be implemented ⚫ Not truly black box, however, as developers have knowledge of implementation ⚫ In practice, developers are likely to use some element of white box as code and tests evolve ⚫ (BDD is clearly black box as it demonstrates behaviour works correctly) 15 Black box testing and TDD ⚫ In TDD, tests are written first ⚫ Executed first (and should fail) ⚫ Act as scaffolding for code so small elements (e.g. single methods) can be tested in isolation as soon as they are coded ⚫ In this sense we are using Black Box testing to develop test cases 16 Boundary of a “unit” ⚫ If we consider a class to be a unit, then it may be completely standalone, or may have relationship to other class(es) ⚫ E.g. an Order class may be related to a Product class ⚫ Testing a method in Order may require Product objects to be created ⚫ This might work in TDD by developing tests then code for Product, then tests and code for Order ⚫ So Product can be included in Order tests – Order “unit” includes Product 17 Mocks and stubs ⚫ Sometimes methods depend on code outwith the unit ⚫ E.g. Order may depend on a remote service that can check on stock levels for a product ⚫ Unit tests may replace those dependencies with mocks or stubs that provide a simplified or simulated response ⚫ E.g. a mock stock level service will simply return a “dummy” value to the Order 18 Designing black box test cases ⚫ This type of testing involves choosing typical extreme & invalid data values which are representative of the specification ⚫ Need a systematic approach ⚫ Don’t just write a few simple tests and consider the unit to be tested ⚫ Can use the following techniques: ⚫ Equivalence partitioning ⚫ Boundary Value Analysis 19 Equivalence partitioning ⚫ Partitions the “Input Domain” of a module under test into classes of similar inputs ⚫ Test data can then be created for each class ⚫ The idea is that a single test case should cover a whole class of input ⚫ Test case design for equivalent partitioning is founded on an evaluation of equivalence classes for an input condition ⚫ An equivalence class depicts a set of valid or invalid states for the input condition 20 Equivalence classes ⚫ Can be defined based on the following types of input condition: Range one valid (within range) and two invalid (below and above) equivalence classes are defined, giving 3 equivalence classes Specific value one valid (equal) and two invalid (below and above) classes are defined, giving 3 equivalence classes Member of a set one valid (member) and one invalid (not member) class is defined, giving 2 equivalence classes Boolean one class is defined for each condition, giving 2 equivalence classes. 21 Equivalence classes example ⚫ Testing a class called Message ⚫ Want to test object can be created if input is valid ⚫ Input parameters and valid values as follows: Area code Blank, or 3,4 or 5 digit number Prefix 3 digit number not beginning with 0 or 1 Suffix 4 digit number Password alphanumeric string (i.e. letters and numbers only) Command “Cheque”, “Deposit”, “Pay Bill” 22 Deriving equivalence classes ⚫ For each data item, the nature of the input condition or conditions (range, value, Boolean, set) is identified ⚫ For each condition, a way must be identified of expressing the specified requirement ⚫ For example, a 3,4 or digit number can be expressed as a range (digits between 3 and 5) ⚫ The Prefix requirement, 3 digits not starting with 0 or 1, could also be expressed as a range (value between 200 and 999), and so on 23 Derived equivalence classes 24 Boundary Value Analysis ⚫ A great many errors occur at the boundaries of the input domain ⚫ For this reason boundary value analysis was developed ⚫ BVA is complementary to equivalence partitioning but concentrates on selection of test cases at the edges of equivalence classes ⚫ BVA also encourages test case designers to look for test cases based on the output conditions 25 Guidelines for BVA if an input condition specifies a range bounded by values a and b, test cases should be designed with values a and b, and values just below a and just above b. if an input condition specifies various values, test cases should be produced to exercise the maximum and minimum numbers, or the first and last members of a set apply guidelines above to output conditions also to develop tests which exercise the output domain in the same way (e.g. if program calculates pay for hourly wage earners, develop test cases to output £0.00 and the maximum, and at least consider how the program might accidentally give an invalid output, i.e a negative amount) if internal program data structures have prescribed boundaries (e.g. arrays), produce test cases to exercise that data structure at its boundary 26 BVA applied to example 27 Selecting test data ⚫ Test data should be selected to cover all of the equivalence classes and include all of the test cases suggested by BVA ⚫ The table on the next slide shows suitable test data for this example ⚫ First test case covers a set of classes/values which should lead to the Accept outcome ⚫ Next test case replaces one value with another valid value in the same class (6a -> 6b) 28 Selecting test data ⚫ Following cases are also variations on first case covering a different case or value within a case each time ⚫ Test cases 1-5 represent valid sets of values ⚫ Test cases 6-13 represent invalid sets of values 29 Test cases 30 Output equivalence classes ⚫ Equivalence class example focused on validity of input ⚫ Testing for correct identification of valid/invalid test cases, not specifically testing for output when input is valid ⚫ This is a focus of the traditional approach to black box testing ⚫ Testing validity of input perhaps less important where user input is limited by UI controls and front end validation ⚫ Can also focus on the output using output equivalence classes 31 Example: triangle problem ⚫ Inputs are triangle side lengths ⚫ Outputs are NotATriangle, Scalene, Isosceles, Equilateral ⚫ Example of equivalence class test cases from Jorgensen, Paul C. Software Testing: A Craftsman’s Approach 32 Example: triangle problem ⚫ This problem can also include test cases for invalid input 33 Incomplete test coverage... “A QA engineer walks into a bar and orders a beer. She orders 2 beers. She orders 0 beers. She orders -1 beers. She orders a lizard. She orders a NULL. She tries to leave without paying. Satisfied, she declares the bar ready for business. The first customer comes in an orders a beer. They finish their drink, and then ask where the bathroom is. The bar explodes” 34 Writing test cases in Java ⚫ Need to write test cases as JUnit test methods ⚫ Could have a separate test method for each test case ⚫ Or, could have a test method for object creation that includes all these test cases ⚫ Or, could have a test method for valid object and a test method for each invalid test case ⚫ Or some other variation ⚫ Choose naming convention to be meaningful among a potentially large list of tests in output from running a test suite 35 JUnit test cases example ⚫ Assume that constructor sets a property valid to true or false depending on input @Test @Test public void validObjectIsCreated() { public void invalidObjectIfAreacode3Digits() { Message m = new Message m = new Message(null,200,1234, "abc123 "," Cheque ”); Message(12,200,1234, "abc123 "," Cheque ”); assertEquals(true, m.isValid()); assertEquals(false, m.isValid()); } m = new Message(null,999,1234, "abc123 "," Cheque ”); @Test assertEquals(true, m.isValid()); public void invalidObjectIfAreacode6Digits() { Message m = new.... Message(123456,200,1234, "abc123 "," Cheque ”); assertEquals(false, m.isValid()); } }... 36 Coming next ⚫ We will look at white box testing in relation to unit testing, and at other phases of testing 37 ⚫ some commentary on mocks/stubs ⚫ https://medium.com/geekculture/software- testing-boundary-value-analysis-using- boundary-value-checking-58febd474188 38 ⚫ IMHO TDD is primarily a whitebox technique, and, I think it is more an implementation technique than a testing technique. When a developer creates code in very small iterations "write test, write code, refactor", it is usually the same person who switches between code and test code, someone who knows exactly the already implemented parts, the missing parts of the "subject under test" as well as the test which are missing. ⚫ When you work that way, notice 39