Handout09.pdf

Teaching Materials - ITX3007/CSX4102 Software Engineering Week#10 – Software Testing Week#10 - Software Testing Topics - Software Testing - Validation and Verification - Testing for development - Test-driven development - Types of software testing Software testing Befor...

Teaching Materials - ITX3007/CSX4102 Software Engineering Week#10 – Software Testing Week#10 - Software Testing Topics - Software Testing - Validation and Verification - Testing for development - Test-driven development - Types of software testing Software testing Before being utilized, software is tested to make sure it functions as intended and to find any flaws. When testing software, artificial data is used to run a program. You review the test run results to check for mistakes, oddities, or details regarding the program's shortcomings. - Can show the existence of mistakes, NOT their absence. - Testing is one of the components of a bigger verification and validation process, which also includes static validation techniques. Aims of software testing - Must prove to the customer and the developer that the product complies with their needs. - This means that each need specified in the requirements document must have at least one test for bespoke software. This means that for generic software products, testing should be done on all system features that will be present in the product release as well as feature combinations. - To spot instances where the software behaves in an incorrect, disagreeable, or non- specified way. - System crashes, data corruption, computation mistakes, and other problematic behaviors, as well as unwanted interactions with other systems, are all targets of defect testing, which tries to find and eliminate them. Testing for validity and defects - Testing for validation is the initial objective. Using a collection of test cases that represent the system's anticipated use, you anticipate the system to operate correctly. - Testing for defects is the second objective. To highlight problems, the test scenarios are designed. Defect testing test cases can even be purposefully ambiguous and don't even need to reflect how the system is commonly used. Test process objectives - Validation testing: proving that the program satisfies the system's criteria to the developer and system customer. An effective test demonstrates that the system functions as planned. - Defect testing: identifying errors or flaws in the software that cause it to behave incorrectly or inconsistently with its specification. A test is successful if it gets the system to behave incorrectly and so reveals a bug in the system. Teaching Materials - ITX3007/CSX4102 Software Engineering Week#10 – Software Testing Figure 10.1. An input-output program testing model Validation versus Verification - "Are we building the product right?" is a verification. o The program must follow its specification. - "Are we building the right product?" is validation. o The software ought to fulfill the actual needs of the user. V&V confidence The goal of V&V is to create trust that the system is "fit for purpose." Based on the system's goal, the user's expectations, and the marketing environment:- - Software objectives: Depending on how important the program is to an organization, the level of confidence varies. - Expectations of users: Users' expectations for some software types may be low. - Environment for marketing: Finding program flaws may not be as crucial as getting a product to market quickly. Testing and inspections In addition to software inspections that analyze the static system representation to look for problems (static verification), tool-based document and code analysis may be utilized. Software evaluation dynamic verification: the act of testing and observing a product's behavior. With test data, the system is run, and its operational behavior is scrutinized. Figure 10.2. Relation of Testing and inspections Teaching Materials - ITX3007/CSX4102 Software Engineering Week#10 – Software Testing Software inspections These entail looking closely at the source representation in order to spot flaws and oddities. Inspections can be utilized prior to implementation because they do not call for system execution. Any kind of system representation, including as requirements, designs, configuration information, test results, etc., can be utilized with them. They have been proven to be a useful method for locating programming faults. The benefits of inspections - Faults can cover up (conceal) other faults during testing. You don't need to worry about relationships between errors as inspection is a static procedure. - A system's incomplete versions can be examined without incurring additional fees. You must create custom test harnesses to evaluate the accessible portions if a program isn't complete. - An inspection can consider a program's more general quality attributes, such as standard compliance, portability, and maintainability, in addition to checking for program defects. Testing and inspections Testing and inspections are complementing verification approaches, not rival ones. When doing the V & V process, both should be used. Inspections can assess adherence to a specification, but they cannot assess adherence to the real demands of the customer. Performance, usability, and other non-functional qualities cannot be examined during inspections. Phases of testing - Development testing is the process of testing a system as it is being created in order to identify issues and defects. - Release testing refers to the process of having a different testing team test a complete system before it is made accessible to users. - During user testing, actual or future users of a system test the system in their own settings. Figure 10.3. Process flow diagram for software testing Teaching Materials - ITX3007/CSX4102 Software Engineering Week#10 – Software Testing Testing for development The term "development testing" refers to all testing activities carried out by the team building the system. - Unit testing: As part of unit testing, certain program modules or object classes are tested. The goal of unit testing should be to confirm how a method or object works. - Component testing: wherein various independent pieces are merged to create composite components. Component interface testing should be the main emphasis of component testing. - System testing: A system's components may need to be integrated in some cases, and the entire system may need to be tested. System testing should place a strong emphasis on component interactions. Unit testing Individual components are tested separately during a unit test. It involves testing for defects. - Individual functions or methods contained within an object. - Classes of objects having numerous properties and methods. A class's whole test coverage entails - testing every action a given object has - Setting and examining every attribute of the object - Exercising the item in every condition that is conceivable. Because the information to be tested is not localized, inheritance makes it more challenging to develop object class tests. - Access to their functionality is provided by composite components with clearly specified interfaces. Figure 10.4. Example of Weather station unit testing Test automation Unit testing should, whenever possible, be automated to allow for quick execution and verification. When performing automated unit testing, you write and execute your program tests using a test automation framework (like JUnit). Frameworks for unit testing offer generic test classes that can be customized to produce special test cases. They can then run each test you've added and give you a report—often via a GUI—on how well it all worked. Teaching Materials - ITX3007/CSX4102 Software Engineering Week#10 – Software Testing Test automation components A setup phase in which the system is initialized with the test case's inputs and anticipated outputs. A call section where the object or method under test is called. A portion of the assertion where the call's outcome is compared to what is anticipated. The test has been successful if the claim evaluates to true; else, it has failed. Unit test cases Selection The test cases should demonstrate that the component you are testing performs as intended when utilized as expected. Test cases should show any flaws in the component if there are any. This results in two categories of unit test cases: The first of these should demonstrate that the component performs as intended and should depict typical program functioning. The other type of test case ought to be focused on assessing knowledge of typical problem areas. To ensure that these are correctly handled and do not cause the component to crash, it should test using atypical inputs. Testing approaches o Using partition testing, you can find input groups that share traits and ought to be handled similarly. Within each of these groupings, you should select tests. o Using testing guidelines to select test cases is known as "guideline-based testing." These recommendations are based on prior knowledge of the kind of mistakes that programmers frequently make when creating components. Partition testing Often, input data and output outcomes belong to different classes with linked members. Each of these classes is an equivalence partition or domain in which every member of the class is treated equally by the program. There should be a unique set of test cases for each partition. Testing guidelines Use sequences with a single value to test software. Use different-sized sequences for various tests. Create tests that allow access to the sequence's first, middle, and last elements. General testing recommendations:- - Select inputs to have the system produce all error warnings. - Design elements that lead to overflowing input buffers - Repeatedly use the same input or set of inputs. - Make sure that invalid outputs are produced - Forced overly large or underlie little outcomes from computation. Teaching Materials - ITX3007/CSX4102 Software Engineering Week#10 – Software Testing Figure 10.5. Equivalence-based partition Component testing Composite components made composed of multiple interacting objects are frequently found in software components. The reconfiguration component, for instance, in the weather station system includes objects that address each reconfiguration-related issue. These objects' functionality is accessible via the specified component interface. Therefore, it's crucial to show that the component interface works as intended while testing composite components. You can assume that all of the unit tests for each of the various objects in a component have been completed. Interface testing The goal is to identify defects brought on by interface mistakes or incorrect interface presumptions. Various interfaces:- - Interfaces for variables: Data exchanged between procedures or methods. - Interfaces for shared memory: Many processes or functions share the same memory block. - Interacts with procedures: A subsystem has a set of resources that other subsystems can access. - Interfaces for passing messages: Subsystems ask other subsystems for services. Teaching Materials - ITX3007/CSX4102 Software Engineering Week#10 – Software Testing Figure 10.6. Interface test relation Interface issues - Abuse of interfaces: When one component calls another, it uses the interface incorrectly, for example, by passing parameters in the wrong sequence. - Interface miscommunication: The calling component has false expectations about how the called component will behave. - Timing issues: Older information is accessible and the called and calling components work at different speeds. Recommendations for interface testing - Create tests with the called procedure's parameters at the very edges of their ranges. - Always use null pointers when testing pointer parameters. - Design evaluations that lead to component failure. - In message-passing systems, use stress testing. - Reorder the components in shared memory systems that are enabled. System testing System testing occurs throughout development when system components are integrated to create a system version, then the united system is tested. System testing's primary objective is to look at how different parts work together. During system testing, compatibility between components, suitable interaction, and prompt data flow across interfaces are all examined. System testing looks at the emergent behavior of a system. System and component testing During system testing, pre-made systems and previously created reusable components may be mixed with newly constructed components. Then, the entire system is evaluated. Now, components created by several subteams or team members can be integrated. System testing is a collective action rather than an individual one. Some businesses may use a separate testing team for system testing without involving designers or programmers. Teaching Materials - ITX3007/CSX4102 Software Engineering Week#10 – Software Testing Use-case testing System testing can be created using the use-cases created to discover system interactions. Testing the use case forces these interactions to happen because each use case often involves multiple system components. The components and interactions that are being evaluated are documented in the sequence diagrams linked to the use case. Figure 10.7. Example of Sequence diagrams are used to create test cases Testing policy Since comprehensive system testing is not achievable, testing policies that specify the minimum level of system test coverage can be created. Testing policies examples:- - It is advisable to evaluate every system feature that can be accessible through a menu. - It is necessary to test combinations of features that can be accessible from the same menu, such as text formatting. - Every function that requires user input must be tested using both right and incorrect input. Test-driven development With test-driven development (TDD), you integrate testing and code development when creating programs. Tests are developed before code, and 'passing' the tests is the key factor in development. You create tests for each increment of code as you create it. The code you have written must pass its test before you can move on to the next increment. TDD was first used in conjunction with agile techniques like Extreme Programming. It can also be applied to development procedures that are plan-driven, though. Teaching Materials - ITX3007/CSX4102 Software Engineering Week#10 – Software Testing Figure 10.8. The process of test-driven development Process actions for TDD - Select the initial functionality increment that is required. Normally, implementing this should only involve a few lines of code. - Create an automated test for this feature and use it in the test. - Execute the test and any other tests that have been added. Since the functionality has not yet been implemented, the new test will fail. - After the functionality has been added, rerun the test. - Following the successful completion of all tests, the following functionality is implemented. Test-driven development advantages - Code protection: All of the code you write has at least one test as each segment of code has an associated test. - Testing for regression: As a program is created, a regression test suite is incrementally created. - Streamlined bug fixing: Where the issue is should be clear when a test fails. It is necessary to review and edit the recently written code. - System specifications: A sort of documentation that outlines what the code ought to be doing is the tests itself. Regression testing Regression testing involves running tests on the system to ensure that no previously functional code has been "broken" by changes. Regression testing is expensive when done manually, but it is easy and uncomplicated when done automatically. Every time the software is altered, all tests are again run. Before a modification is committed, tests must ’successfully' complete their runs. Release testing Release testing is the process of evaluating a specific system release that is intended for use by users outside the development team. The main objective of the release testing process is to convince the system's creator that the end product is safe for implementation. Therefore, release testing must show that the system satisfies it’s functional, performance, and reliability and is reliable under regular operating settings. Teaching Materials - ITX3007/CSX4102 Software Engineering Week#10 – Software Testing Release testing is frequently a type of "black-box" testing in which only tests created from the system specification are engaged. Release testing and system testing System testing includes release testing. Important distinctions: - - A distinct team that wasn't involved in the system's development should undertake release testing. - The development team should focus their system testing efforts on identifying systemic flaws (defect testing). The purpose of release testing (validation testing) is to verify that the system satisfies its specifications and is suitable for usage by the wider public. Requirements based testing Requirements-based testing is the process of creating a test or tests for each need. Conditions for the Mentcare system:- - If a patient has a history of allergies to a specific medicine, the system user will receive a warning notice if that medication is prescribed. - A prescriber must explain why they choose to disregard an allergy warning if they do so. Figure 10.9. Example of Requirement test in Mentcare Performance testing As part of release testing, a system's emergent properties, such as reliability and performance, may be examined. Tests should accurately follow the system's use profile. Performance tests are often planned as a set of tests where the amount is progressively added until the system performance is insufficient. During a performance test, a system's failure behavior is investigated when it is purposefully overloaded. User testing The "user or customer testing" phase of the testing process is where customers or users provide comments and suggestions on system testing. Even after thorough system and release testing has been done, user testing is still crucial. Teaching Materials - ITX3007/CSX4102 Software Engineering Week#10 – Software Testing This is because a system's performance, usability, dependability, and robustness are all greatly influenced by factors related to the user's workplace. These are impossible to duplicate in a testing setting. User testing types - Alpha Test: Together with the development team, users test the program at the developer's facility. - Beta Test: Users are provided access to a program release so they can experiment with it and talk to the system developers about any problems they encounter. - Acceptance Test: Customers review a system after the system developers have given it their seal of approval to decide whether or not it is ready for deployment in the customer environment. Figure 10.10. The procedure for acceptability testing Acceptance testing and agile methodologies In agile approaches, the user/customer participates in the development team and is in charge of deciding whether the system is acceptable. The user or client defines the tests, which are interconnected with other tests in that they are launched automatically in response to changes. There isn't a separate process for acceptability testing. The primary issue in this case is whether or not the embedded user is "typical" and is able to speak for all system stakeholders. Summary - Testing is limited to identifying program errors. It cannot prove that all flaws have been fixed. - The software development team is in charge of development testing. Before a system is made available to clients, it should be tested by a different team. - Testing during development includes unit testing, which involves testing individual items, component testing, which involves testing linked groups of objects, and system testing, which involves testing partial or full systems. - To "break" software when testing it, employ test scenarios that have been successful in identifying errors in other systems.. Teaching Materials - ITX3007/CSX4102 Software Engineering Week#10 – Software Testing - Whenever possible, it should be done to write automated tests. A piece of software that can be used each time a system is modified contains the tests. - Tests are written before the code that will be tested in the test-first approach to development. - In scenario testing, test cases are created by imagining a typical usage scenario. - In order to determine whether the program is suitable for deployment and use in its operating environment, a user testing procedure called acceptance testing is used.