Software Engineering II - Lecture Notes - Software Testing (Unit & Integration)

Software Engineering II Presented by Dr. Amira M. Gaber Lecture III: Software Testing (Unit Test)  What is the Software Testing - Validation Testing? The process of evaluating software during the development process or at the end of the developm...

Software Engineering II Presented by Dr. Amira M. Gaber Lecture III: Software Testing (Unit Test)  What is the Software Testing - Validation Testing? The process of evaluating software during the development process or at the end of the development process to determine whether it satisfies specified business requirements. Validation Testing ensures that the product actually meets the client's needs. It can also be defined as to demonstrate that the product fulfills its intended use when deployed on appropriate environment. It answers to the question, Are we building the right product?  Validation Testing - Workflow Validation testing can be best demonstrated using V-Model. The Software/product under test is evaluated during this type of testing. Activities: Unit Testing Integration Testing System Testing User Acceptance Testing  Unit Test Unit Testing is a type of software testing where individual units or components of a software are tested. The purpose is to validate that each unit of the software code performs as expected. Unit Testing is done during the development (coding phase) of an application by the developers. Unit Tests isolate a section of code and verify its correctness. A unit may be an individual function, method, procedure, module, or object.  Unit Test In SDLC, Unit testing is first level of testing done before integration testing. Unit testing is a White Box testing technique that is usually performed by the developer.  Why perform Unit Testing? Unit Testing is important because software developers try saving time ,because inappropriate unit testing leads to high cost Defect fixing during System Testing, Integration Testing and even Beta Testing after application is built. If proper unit testing is done in early development, then it saves time and money in the end.  Why perform Unit Testing? 1. Unit tests help to fix bugs early in the development cycle and save costs. 2. It helps the developers to understand the testing code base and enables them to make changes quickly 3. Good unit tests serve as project documentation 4. Unit tests help with code re-use. Migrate both your code and your tests to your new project.  How to execute Unit Testing In order to execute Unit Tests, Unit Testing is of two types Manual: the unit testing may employ a step-by-step instructional document. Automated: the developers write a section of code to test a specific function in software application. Developers can also isolate this function to test more rigorously which reveals unnecessary dependencies between function being tested and other units so the dependencies can be eliminated. Developers generally use Unit Test framework to develop automated test cases for unit testing.  Unit Testing Techniques The Unit Testing Techniques are mainly categorized into three parts which are Black box testing that involves testing of user interface along with input and output, White box testing that involves testing the functional behavior of the software application and Gray box testing that is used to execute test suites, test methods, test cases and performing risk analysis.  Code coverage techniques in Unit Test Code coverage techniques used in Unit Testing are listed below: Statement Coverage Decision Coverage Branch Coverage Condition Coverage Finite State Machine Coverage  1. Code coverage techniques in Unit Test Statement Coverage is a white box testing technique in which all the executable statements in the source code are executed at least once. It is used for calculation of the number of statements in source code which have been executed. The main purpose of Statement Coverage is to cover all the possible paths, lines and statements in source code. Statement Coverage Result  2. Decision Coverage Decision Coverage is a white box testing technique which reports the true or false outcomes of each Boolean expression of the source code. The goal of decision coverage testing is to cover and validate all the accessible source code by checking and ensuring that each branch of every possible decision point is executed at least once. Decision Coverage  3. Branch Coverage Branch Coverage is a white box testing method in which every outcome from a code module(statement or loop) is tested. The purpose of branch coverage is to ensure that each decision condition from every branch is executed at least once. It helps to measure fractions of independent code segments and to find out sections having no branches. Branch Coverage  Branch Coverage Advantages Allows you to validate-all the branches in the code Helps you to ensure that no branched lead to any abnormality of the program’s operation Branch coverage method removes issues which happen because of statement coverage testing Allows you to find those areas which are not tested by other testing methods Branch coverage ignores branches inside the Boolean expressions  4. Condition Coverage Condition Coverage or expression coverage is a testing method used to test and evaluate the variables or sub-expressions in the conditional statement. The goal of condition coverage is to check individual outcomes for each logical condition. In this coverage, expressions with logical operands are only considered.  5. Finite State Machine Coverage Finite state machine coverage is certainly the most complex type of code coverage method. This is because it works on the behavior of the design. In this coverage method, you need to look for how many time-specific states are visited, transited. It also checks how many sequences are included in a finite state machine  Unit Testing Advantages Developers looking to learn what functionality is provided by a unit and how to use it can look at the unit tests to gain a basic understanding of the unit API. Unit testing allows the programmer to refactor code at a later date, and make sure the module still works correctly (i.e. Regression testing). The procedure is to write test cases for all functions and methods so that whenever a change causes a fault, it can be quickly identified and fixed. Due to the modular nature of the unit testing, we can test parts of the project without waiting for others to be completed.  Unit Testing Disadvantages Unit testing can’t be expected to catch every error in a program. It is not possible to evaluate all execution paths even in the most trivial programs Unit testing by its very nature focuses on a unit of code. Hence it can’t catch integration errors or broad system level errors.  Unit Testing Best Practices Unit Test cases should be independent. In case of any enhancements or change in requirements, unit test cases should not be affected. Test only one code at a time. Follow clear and consistent naming conventions for your unit tests In case of a change in code in any module, ensure there is a corresponding unit Test Case for the module, and the module passes the tests before changing the implementation Bugs identified during unit testing must be fixed before proceeding to the next phase in SDLC Adopt a “test as your code” approach. The more code you write without testing, the more paths you have to check for errors. Software Engineering II Presented by Dr. Amira M. Gaber Lecture III: Software Testing (Integration Test)  Validation Testing - Workflow Validation testing can be best demonstrated using V-Model. The Software/product under test is evaluated during this type of testing. Activities: Unit Testing Integration Testing System Testing User Acceptance Testing  Integration Test Integration Testing Integration testing -- also known as integration and testing (I&T) -- is a type of software testing in which the different units, modules or components of a software application are tested as a combined entity. However, these modules may be coded by different programmers.  Integration Test The aim of integration testing is to test the interfaces between the modules and expose any defects that may arise when these components are integrated and need to interact with each other.  What does integration testing involve? Integration testing involves integrating the various modules of an application and then testing their behavior as a combined, or integrated, unit. Verifying if the individual units are communicating with each other properly and working as intended post-integration is essential.  Integration testing versus unit testing Integration testing is conducted after unit testing, where the functional correctness of the smallest piece of code, or unit, is tested. Each unit can be logically isolated in the software. The smaller the unit, the more granular insights unit testing can reveal.  Difference between unit testing and integration testing The unit testing, the individual modules are tested. In integration testing, these modules are combined and tested as a single unit to check the functionality of the overall application.  Why integration testing is essential 1. Integrating disparate modules into a working application: When different developers work on different modules, individuals bring their own understanding and logic to the development effort. This can cause functional or usability problems when the modules are combined. Integration testing can help ensure that the integrated units function properly as one unit and align with stated requirements. It can also ensure there are no errors between the different interfaces of different modules.  Why integration testing is essential 2. Ensuring that changing requirements are incorporated into the application: In many real-time application scenarios, requirements can and do change often. These new requirements may not be unit-tested every time, which may lead to missed defects or missing product features. Integration testing can fill in these gaps to ensure that new requirements are incorporated into the final application.  Why integration testing is essential 3. Eliminating common issues missed during unit testing: Some modules that interact with third-party application program interfaces (APIs) need to be tested to ensure they function properly. This may not be done during unit testing, so integration testing is required. 4. Eliminating other common problems: Integration testing also helps eliminate issues, such as inadequate exception handling, API response generation, data formatting, erroneous external hardware interfaces, incorrect third-party service interfaces and error trapping.  Who is responsible for integration testing? The testers are responsible for system integration testing, particularly the end-to–end test. Testers often ask the product owner or business analyst for help in developing the test scenarios and reviewing the cases.  Integration Testing Approaches Integration testing Approaches consist of Big Bang Approach : Incremental Approach: which is further divided into the following Top Down Approach Bottom Up Approach Sandwich Approach – Combination of Top Down and Bottom Up  Big Bang Testing Big Bang Testing is an Integration testing approach in which all the components or modules are integrated together at once and then tested as a unit. This combined set of components is considered as an entity while testing. If all of the components in the unit are not completed, the integration process will not execute.  Big Bang Testing Advantages Its suitability for testing small systems. Its ease of identifying errors in such systems, saving time and speeding up application deployment.  Big Bang Testing Disadvantages Fault Localization is difficult. Given the sheer number of interfaces that need to be tested in this approach, some interfaces link to be tested could be missed easily. Since the Integration testing can commence only after “all” the modules are designed, the testing team will have less time for execution in the testing phase. Since all modules are tested at once, high-risk critical modules are not isolated and tested on priority. Peripheral modules which deal with user interfaces are also not isolated and are tested on priority.  Incremental Testing In the Incremental Testing approach, testing is done by integrating two or more modules that are logically related to each other and then tested for proper functioning of the application. Then the other related modules are integrated incrementally and the process continues until all the logically related modules are integrated and tested successfully. Incremental Approach, in turn, is carried out by two different Methods: Bottom Up Top Down  Incremental Testing(Bottom-up Integration Testing) Bottom-up Integration Testing is a strategy in which the lower level modules are tested first. These tested modules are then further used to facilitate the testing of higher level modules. The process continues until all modules at top level are tested. Once the lower level modules are tested and integrated, then the next level of modules are formed.  Incremental Testing(Bottom-up Integration Testing) Advantages: Fault localization is easier. No time is wasted waiting for all modules to be developed unlike Big-bang approach Disadvantages: Critical modules (at the top level of software architecture) which control the flow of application are tested last and may be prone to defects. An early prototype is not possible  Incremental Testing(Top-up Integration Testing) Top Down Integration Testing is a method in which integration testing takes place from top to bottom following the control flow of software system. The higher level modules are tested first and then lower level modules are tested and integrated in order to check the software functionality. Stubs are used for testing if some modules are not ready.  Incremental Testing(Top-up Integration Testing) Advantages: Fault Localization is easier. Possibility to obtain an early prototype. Critical Modules are tested on priority; major design flaws could be found and fixed first. Disadvantages: Needs many Stubs. Modules at a lower level are tested inadequately.  Incremental Testing(Sandwich Testing) Sandwich Testing is a strategy in which top level modules are tested with lower level modules at the same time lower modules are integrated with top modules and tested as a system. It is a combination of Top-down and Bottom-up approaches therefore it is called Hybrid Integration Testing.  How to do Integration Testing? The Integration test procedure irrespective of the Software testing strategies (discussed above): Prepare the Integration Tests Plan Design the Test Scenarios, Cases, and Scripts. Executing the test Cases followed by reporting the defects. Tracking & re-testing the defects. Steps 3 and 4 are repeated until the completion of Integration is successful. Brief Description of Integration Test Plans It includes the following attributes: Methods/Approaches to testing (as discussed above). Scopes and Out of Scopes Items of Integration Testing. Roles and Responsibilities. Pre-requisites for Integration testing. Testing environment. Risk and Mitigation Plans.  Input and Output of Integration Testing Input: Unit Tested Components/Modules All High prioritized bugs fixed and closed All Modules to be code completed and integrated successfully. Integration tests Plan, test case, scenarios to be signed off and documented.  Input and Output of Integration Testing Output Successful Testing of Integrated Application. Executed Test Cases are documented All High prioritized bugs fixed and closed Technical documents to be submitted followed by release Notes. Best Practices/ Guidelines for Integration Testing First, determine the Integration Test Strategy that could be adopted and later prepare the test cases and test data accordingly. Study the Architecture design of the Application and identify the Critical Modules. These need to be tested on priority. Obtain the interface designs from the Architectural team and create test cases to verify all of the interfaces in detail. Interface to database/external hardware/software application must be tested in detail. After the test cases, it’s the test data which plays the critical role. Always have the mock data prepared, prior to executing. Do not select test data while executing the test cases. Software Engineering II Presented by Dr. Amira M. Gaber Lecture III: Software Testing (System Test)  Validation Testing - Workflow Validation testing can be best demonstrated using V-Model. The Software/product under test is evaluated during this type of testing. Activities: Unit Testing Integration Testing System Testing User Acceptance Testing  System Testing System Testing is a type of software testing that is performed on a complete integrated system to evaluate the compliance of the system with the corresponding requirements. In system testing, integration testing passed components are taken as input.  System Testing System Testing is a black-box testing. System Testing is performed after the integration testing and before the acceptance testing.  Goal of System Testing The goal of System testing detects defects within both the integrated units and the whole system. The result of system testing is the observed behavior of a component or a system when it is tested.  What does System testing involve? System Testing is carried out on the whole system in the context of either system requirement specifications or functional requirement specifications or in the context of both. System testing tests the design and behavior of the system and also the expectations of the customer. It is performed to test the system beyond the bounds mentioned in the software requirements specification (SRS).  Who is responsible for System testing? System Testing is basically performed by a testing team that is independent of the development team that helps to test the quality of the system impartial. It has both functional and non-functional testing.  System Testing Process System Testing Process: System Testing is performed in the following steps: Test Environment Setup: Create testing environment for the better quality testing. Create Test Case: Generate test case for the testing process. Create Test Data: Generate the data that is to be tested. Execute Test Case: After the generation of the test case and the test data, test cases are executed.  System Testing Process System Testing Process: System Testing is performed in the following steps: Defect Reporting: Defects in the system are detected. Regression Testing: It is carried out to test the side effects of the testing process. Log Defects: Defects are fixed in this step. Retest: If the test is not successful then again test is performed.  Types of System Testing Performance Testing: Performance Testing is a type of software testing that is carried out to test the speed, scalability, stability and reliability of the software product or application. Load Testing: Load Testing is a type of software Testing which is carried out to determine the behavior of a system or software product under extreme load. Stress Testing: Stress Testing is a type of software testing performed to check the robustness of the system under the varying loads. Scalability Testing: Scalability Testing is a type of software testing which is carried out to check the performance of a software application or system in terms of its capability to scale up or scale down the number of user request load.  Advantages of System Testing The testers do not require more knowledge of programming to carry out this testing. It will test the entire product or software so that we will easily detect the errors or defects which cannot be identified during the unit testing and integration testing. The testing environment is similar to that of the real time production or business environment. It checks the entire functionality of the system with different test scripts and also it covers the technical and business requirements of clients. After this testing, the product will almost cover all the possible bugs or errors and hence the development team will confidently go ahead with acceptance testing.  Disadvantages of System Testing This testing is time consuming process than another testing techniques since it checks the entire product or software. The cost for the testing will be high since it covers the testing of entire software. It needs good debugging tool otherwise the hidden errors will not be found. Software Engineering II Presented by Dr. Amira M. Gaber Lecture III: Software Testing (Acceptance Test)  Validation Testing - Workflow Validation testing can be best demonstrated using V-Model. The Software/product under test is evaluated during this type of testing. Activities: Unit Testing Integration Testing System Testing User Acceptance Testing  Acceptance Testing It is a formal testing according to user needs, requirements and business processes conducted to determine whether a system satisfies the acceptance criteria or not and to enable the users, customers or other authorized entities to determine whether to accept the system or not.  Acceptance Testing Acceptance Testing is the last phase of software testing performed after System Testing and before making the system available for actual use.  Types of Acceptance Testing User Acceptance Testing (UAT): User acceptance testing is used to determine whether the product is working for the user correctly. Specific requirements which are quite often used by the customers are primarily picked for the testing purpose. This is also termed as End-User Testing. Business Acceptance Testing (BAT): BAT is used to determine whether the product meets the business goals and purposes or not. BAT mainly focuses on business profits which are quite challenging due to the changing market conditions and new technologies so the current implementation may have to being changed which results in extra budgets.  Types of Acceptance Testing Contract Acceptance Testing (CAT): CAT is a contract that specifies that once the product goes live, within a predetermined period, the acceptance test must be performed and it should pass all the acceptance use cases. Here is a contract termed a Service Level Agreement (SLA), which includes the terms where the payment will be made only if the Product services are in-line with all the requirements, which means the contract is fulfilled. Sometimes, this contract happens before the product goes live. There should be a well-defined contract in terms of the period of testing, areas of testing, conditions on issues encountered at later stages, payments, etc.  Types of Acceptance Testing Regulations Acceptance Testing (RAT): RAT is used to determine whether the product violates the rules and regulations that are defined by the government of the country where it is being released. This may be unintentional but will impact negatively on the business. Generally, the product or application that is to be released in the market, has to go under RAT, as different countries or regions have different rules and regulations defined by its governing bodies. If any rules and regulations are violated for any country then that country or the specific region then the product will not be released in that country or region. If the product is released even though there is a violation then only the vendors of the product will be directly responsible. Operational Acceptance Testing (OAT): OAT is used to determine the operational readiness of the product and is non-functional testing. It mainly includes testing of recovery, compatibility, maintainability, reliability, etc. OAT assures the stability of the product before it is released to production.  Types of Acceptance Testing Alpha Testing: Alpha testing is used to determine the product in the development testing environment by a specialized testers team usually called alpha testers. Beta Testing: Beta testing is used to assess the product by exposing it to the real end-users, usually called beta testers in their environment. Feedback is collected from the users and the defects are fixed. Also, this helps in enhancing the product to give a rich user experience.  Use of Acceptance Testing To find the defects missed during the functional testing phase. How well the product is developed. A product is what actually the customers need. Feedback help in improving the product performance and user experience. Minimize or eliminate the issues arising from the production.  Advantages of Acceptance Testing This testing helps the project team to know the further requirements from the users directly as it involves the users for testing. Automated test execution. It brings confidence and satisfaction to the clients as they are directly involved in the testing process. It is easier for the user to describe their requirement. It covers only the Black-Box testing process and hence the entire functionality of the product will be tested.  Disadvantages of Acceptance Testing Users should have basic knowledge about the product or application. Sometimes, users don’t want to participate in the testing process. The feedback for the testing takes long time as it involves many users and the opinions may differ from one user to another user. Development team is not participated in this testing process.