Software Testing: Functional and Unit Testing

• Software testing, validation, and verification is the context of the following notes
• Dr. Yasmine Afify presents the lectures

Software Testing Types

• Functional Testing
• Non-functional Testing
• Structural/White-box Testing
• Change-related Testing covers Re-testing and Regression

Functional/Dynamic Testing

• It involves designing test cases, preparing test data, running a program with test data, and comparing the results to test cases
• It tests and evaluates functions that a system should perform
• Every test level of suitable test environment is needed
• Acceptance testing requires a production-like environment
• Component testing allows developers to use their own environment

Component/unit/module testing

• This process tests individual components in isolation, and is carried out by the team developing the system
• Components tested can be individual functions, methods within an object, classes, and database modules
• Examples of defects found are incorrect functionality, data flow problems, and incorrect code and logic

Advantages of Unit Testing

• Compound errors are less likely the earlier a problem is identified
• Fixing problems early is usually cheaper than fixing them later in development
• It makes debugging processes easier
• Developers can quickly make changes to the codebase
• Developers can reuse code and migrate it to new projects

Integration testing

• This is a software testing level where individual units are combined and tested as a group
• Its purpose is exposing interaction faults between integrated units
• Takes into account the broader context of the application: database, external APIs, filesystem
• Interfacing errors not detected in unit testing may appear
• Timing problems in real-time systems can be detected, where they are not detectable by unit testing
• Assumes that component tests on the individual objects within the component have been completed
• Integration tests require a deployment step, whereas unit tests should run at build-time and should be quick to run

Integration Testing Sample Test Cases

• Test case has an application with 3 modules which are Login Page, Mailbox, and Delete Emails
• Do not concentrate much on the Login Page testing as it's already been done in Unit Testing.
• Check how the Login Page is linked to the Mail Box Page.
• Mailbox is tested for Check how it is integrated to the Delete Mails Module.

Integration Testing Test Case Example

• Test Case ID 1 is to check the interface link between the Login and Mailbox module
• The test case description for ID 1 is to enter login credentials and click on the Login button
• The expected result from test case ID 1 is to be directed to the Mail Box
• Test Case ID 2 is to check the interface link between the Mailbox and Delete Mails Module
• The test case description for ID 2 is from Mailbox select the email and click a delete button
• The expected result from test case ID 2 is the selected email should appear in the Deleted/Trash folder

Integration Testing Types

• Decomposition-based Integration
• Call Graph-Based Integration
• Path-Based Integration

Decomposition-based Integration: Big-bang Integration

• Approach waits until all the components arrive, and only one round of integration testing is done
• The integration procedure will not proceed until all the elements have been integrated
• The components or modules are integrated together at once and then tested as a unit
• Combined set of components is considered as an entity while testing
• It is ideal for a product where the interfaces are stable with few number of defects

Advantages of Big-bang Integration

• It is convenient for small systems
• The amount of planning needed for this type of testing is almost non-existent

Disadvantages of Big-bang Integration

• Fault Localization is difficult, detaching all relevant modules to investigate its underlying cause may prove challenging
• Given the large number of interfaces, there is a chance that some interfaces link to be tested could be missed easily
• Integration testing can only commence after "all" the modules are designed, giving the testing team less time in testing phase
• High-risk critical modules are not isolated and tested on priority since all modules are tested at once
• Higher chance of failure since all the modules are tested together

Stubs and Drivers

• Stubs and Drivers are the dummy programs in integration testing used to substitute for the missing modules
• They simulate data communication with the calling module without implementing the entire programming logic of the replaced module
• Stub is called by the Module under Test; it contains signature of methods and additional print/display statement to confirm it works
• Driver calls the Module to be tested; it contains code to call the other module and pass it appropriate parameters
• The test is repeated when stub/driver is replaced by the actual code, which costs additional testing time and cost

Decomposition-based Integration - Bottom-up Integration Approach

• Testing is strategy in which the lower-level modules are tested first.
• Use the tested modules to facilitate the testing of higher-level modules
• The process continues until all modules at top level are tested

Advantages to Bottom-up Integration Approach

• Fault localization is easy to achieve
• Do not waste time waiting for all modules to be developed (unlike Big-bang approach)

Disadvantages to Bottom-up Integration Approach

• Critical modules at the top level of software architecture which control the flow of application are tested last and may be prone to defects
• Early prototype is not possible if UI testing is done at last
• Needs many drivers

Decomposition-based Integration - Top-down Integration Approach

• Method in which integration testing takes place from top to bottom following the control flow of software system
• Higher-level modules are tested first, and then lower-level modules are tested and integrated to check software functionality
• Stubs are used for testing if some modules are not ready
• Testing is first/depth first traversal

Advantages of Top-down Integration Approach

• Fault localization is easy.
• Critical modules are tested on priority; major design flows could be found and fixed first.
• Possibility to obtain an early prototype.

Disadvantages of Top-down Integration Approach

• Needs many stubs.
• The basic functionalities of the software are tested at the end of the cycle.

Decomposition-based Integration - Sandwich/Hybrid Integration Approach

• Strategy in which the system is categorized into three layers with the middle layer as the target
• Testing is performed in parallel in both directions and is grouped in the middle layer
• It is a combination of Top-down and Bottom-up approaches, therefore it is called Hybrid Integration Testing

Advantages of Sandwich Integration Approach

• This can reduce the time and effort of testing the system as a whole as it can identify and fix the integration errors earlier and faster.
• This can facilitate the collaboration and feedback between the developers and testers, as it can expose the dependencies and interfaces of the modules.

Disadvantages of Sandwich Integration Approach

• There requires a lot of coordination and communication between the developers and testers who work on different levels of the system.
• This may involve creating and maintaining many stubs and drivers.
• It may not detect all the integration errors, especially those that occur in the intermediate modules, which are tested last.

System testing

• It focuses on a complete, integrated system to evaluate compliance with specified requirements
• Tests are made on characteristics only present when the entire system is run
• Evaluates the end-to-end system specifications
• Focuses on non-functional aspects (quality factors) of the product
• Tests product behaviour in a complete and realistic environment, corresponds to the final production
• Independent testers typically carry out it testing

Test Objects for System Testing

• Applications
• Hardware/software systems
• Operating systems
• System under test (SUT)
• System configuration and configuration data

Typical Defects for System Testing

• Includes incorrect or unexpected system functional or non-functional behaviour
• Failure to properly and completely carry out end-to-end functional tasks
• Failure of the system to work properly in the production environment(s)
• Failure of the system to work as described in system and user manuals

Release testing

• Testing a release of a system intended for use outside of development team
• To convince the supplier of the system that it is good enough for use
• Shows the system delivers its specified functionality, performance, and dependability, and that it does not fail during normal use
• Is usually a black-box testing process where tests are only derived from the system specification

Differences from System Testing

• A separate team that has not been involved in the system development should be responsible for it
• Testing by the development team should focus on discovering bugs in the system (defect testing)
• The objective of testing is to check that the system meets its requirements and is good enough for external use (validation testing)

Smoke Testing

• Type of software testing with a non-exhaustive set of tests that aim at ensuring the most critical/important functions work
• It is executed "before" any detailed functional or regression tests are executed on the software build
• Its purpose is to reject a badly broken application so that the Quality team does not waste time installing and testing the software application
• Used to decide if a build is stable enough to proceed with further testing
• Is conducted by developers or testers

Examples of smoke test scenarios

• Web Applications: Verify that the login/authentication process is working correctly
• Web Applications: Access the main pages and make sure they load without errors
• Web Applications: Submit a form and validate successful data submission
• Web Applications: Perform basic navigation and confirm smooth transitions between pages
• Mobile App: Install the app and make sure it launches without any crashes
• Mobile App: Navigate through the core screens and validate their responsiveness
• Mobile App: Perform required interactions such as button clicks or form inputs
• Mobile App: Verify functionality of key features such as push notifications or location services

Sanity Testing

• Conducted for a build with minor bug fixes or new code added
• Targets specific areas affected by bug fixes or new code
• If rejected, then the build won't proceed to further testing

Acceptance Testing Types

• (UAT) User Acceptance Testing
• Business Acceptance Testing
• Contract Acceptance Testing
• Operational Acceptance Testing (OAT)
• Regulation Acceptance Testing (RAT)
• Alpha Testing
• Beta Testing

User acceptance testing (UAT)

• This is a stage in the testing process with users who provide input and advice on system testing
• Main objective is building confidence that users can use the system to meet their needs, fulfil requirements, and perform business processes with minimum difficulty, cost, and risk
• It is important because users have different perspective than the developers
• User’s working environment have a major effect on the reliability, performance, usability and robustness of a system; these cannot be replicated in a testing environment

More on (UAT)

• The quality team has a meeting with the client, with "UAT test cases" which are the basic scenarios the client should run himself
• Afterwards the client will give feedback in the form of bugs or approval
• There is a sign off that "UAT has passed successfully which is a very crucial activity done for all projects in all IT companies and the quality team is responsible for managing it

Alpha and Beta Testing

• Used for commercial off-the-shelf (COTS) software to get feedback from potential/existing users/customers before putting the software product on the market
• Production team tests the software in a lab environment at the developer's site in alpha testing
• After alpha testing is done, a beta version of the software is made available to users to experiment and raise problems in their environment

Operational Acceptance Testing (OAT)

• Acceptance testing of the system by operations/administration staff is usually performed in a (simulated) production environment
• Tests focus on operational aspects such as backup/restore; installing, uninstalling/upgrading; disaster recovery; user management; and maintenance tasks

Contractual acceptance testing (CAT)

• Performed against a contract's acceptance criteria for producing custom-developed software
• Acceptance criteria should be defined when the parties agree to the contract

Regulatory acceptance testing (RAT)

• Performed against any regulations that must be adhered to, such as government, legal, or safety regulations
• If it undergoes all the rules and regulations, it is RAT
• Not even a single violation has to be found while this the testing: if any occurs, the whole product will be is failure, making this the most important and complicated testing type

Test Levels

• Done at different points in the development life cycle
• Performed on a different object in a system work product
• Has different objectives and requirements
• Refers to a group of test activities that are organized & managed together

Integration Testing Detailed

• Done at component level, after system testing
• Tests interfaces between items to concentrate on the interactions between items, rather than functionality
• Tests can involve both functional & structural approaches

System Testing Detailed

• Is concerned with the behavior of the whole product
• Has a test environment is that should represent live
• Includes end-to-end, business process testing
• Involves functional and non-functional testing
• Often done by independent test team

Acceptance testing Detailed

• Typically done by customers and / or users
• Main goal is to establish confidence in the software
• Finding defects is not the main focus
• Four subtypes as previously mentioned