Program Testing and Goals

Program Testing

• Testing aims to verify intended program behavior
• Testing can also discover defects before deployment
• During software testing, a program is executed using artificial or generated data.
• The results of these tests are analyzed for errors, anomalies, and non-functional attribute information.
• Testing can reveal the presence of errors, but it cannot guarantee their complete absence.
• Testing is a component of verification and validation processes and includes static validation techniques

Program Testing Goals

• Testing aims to demonstrate compliance with requirements and to discover defects
• For custom software, the goal is to ensure that every requirement is tested.
• For generic software products, the goal is to verify all system features function as intended.
• Defect finding through software testing can detect incorrect, undesirable, or non-conforming software behavior.
• Defect testing targets system crashes, incorrect computations, and data corruption.

Validation Testing

• Validates that the system performs correctly and meets requirements under typical use scenarios.
• The focus is to ensure that software behaves as expected, aligning with real-world conditions and specified requirements.
• For Example, validating a banking app with standard transactions, like deposits and withdrawals ensures actions are processed correctly.

Defect Testing

• Locates defects through test cases that target hidden issues.
• Scenarios will be unconventional or extreme.
• Detects incorrect, undesirable, or non-conforming behavior not evident in normal use.
• Inputting deliberate unexpected data can check if the system handles situations gracefully or by failing.

Testing Process Goals: Validation

• The testing aim is to verify that the software operation is as intended.
• The test is successful when the system performs correctly and aligns with requirements

Testing Process Goals: Defect

• The objective is to identify faults where the system deviates from its specifications.
• Tests will be considered successful if they expose a defect that causes incorrect system operation or failure to meet requirements.

Verification

• Verification is confirming that the software should conform to its specifications.

Validation

• Validation is confirming that the software should do what the user really requires.

V&V (Verification and Validation) Confidence

• V&V are critical for ensuring a system meets requirements and is fit for its intended use.
• The primary V&V goal is establishing confidence that a software system meets requirements
• Confidence is developed through rigorous testing and evaluation.

V&V Confidence Factors

• The software purpose is a factor, the confidence level will depend on the organization.
• User expectations, users may have low expectations for certain types of software.
• Marketing Environment, early releases can be more important than defect removal.

Software Inspections

• Static verification analyzes system documentation, design, or code without execution.
• The goal is to identify defects, inconsistencies, or potential improvements through detailed examination.
• Inspections and testing ensuring software reliability
• Inspections do not require system execution.
• Inspections involve examining source code to find anomalies or defects.
• They can be applied to system representations such as requirements, design, and test data.
• Inspections effectively discover program errors.

Advantages of Inspections

• Inspections do not include interactions between errors, preventing masking.
• The inspection can consider quality attributes like standards, portability and maintainability.
• Incomplete versions of the system can be inspected without additional costs, and will not need specialized test harnesses

Inspections and Testing Compared

• Inspections and testing are complementary verification techniques that should both be used during the V&V process
• Inspections cannot check non-functional characteristics like performance or usability.
• Inspections can verify conformance to specifications but not to real customer requirements.

Stages of Testing

• There are three primary stages of testing that ensure software is robust, reliable, and ready for end-users.
• Development : testing activities are conducted during development to identify and correct defects.
• Release : The stage where a complete version of the software is tested by a separate testing team.
• User : Actual or potential users test software in their environments to gather feedback.

Types of Development Testing

• Development testing identifies and fixes defects early in the development cycle.
• The main types are Unit, Component and System testing

Unit Testing

• Focuses on testing individual program units or object classes in isolation
• The function is to verify the functionality of specific objects or methods in isolation

Component Testing

• Focuses on testing composite components formed by integrating several individual units
• The function is to validate the interactions and interfaces between integrated components.

System Testing

• System testing includes all components including complete and integrated
• They ensure the system functions correctly, with a focus on correct interactions between components
• Each testing level builds upon previous, ensuring comprehensive validation

Unit Testing Details

• Unit testing is a defect-testing process that tests individual components in isolation.
• Units could testing individual functions, object classes, or composite components with defined interfaces.

Object Class Testing

• Object classes undergo comprehensive coverage via feature verification, attribute checking, and state change simulation.
• Testing should test all testing features.
• Verify all attribute values to confirm correct behavior.
• Inheritance increases the difficulty of object class testing because the information is not localized

Weather Station Testing

• Comprehensive testing is crucial for its various functions and all possible states and transitions must be covered.
• Key functions to test include reportWeather to ensure accurate weather reports per current data.
• The process must calibrate and verify of all instruments.
• Self testing functions are important, including starting properly with correct state transitions.
• The weather station must shutdown correctly, with proper clean up.

Weather Station: State Model Testing

• State Model testing includes identifying and documenting system states and transitions.
• Events should be simulated to test the system response to state changes.
• Example can include testing transitions between the Shutdown, Running, Testing, and Transmitting.
• All transitions should be checked

Automated Testing

• Automated testing uses tools and frameworks for testing to run and check itself without manual intervention.
• Utilize test automation frameworks like JUnit to write and execute tests.
• Frameworks should be automated to ensure they are run consistently and efficiently.
• Frameworks can run tests and report outcomes via GUI.

Automated Test Components

• Automated Tests follow a basic structure with Setup, Call and Assertion steps
• Setup: Initializes the system with necessary inputs and expected outputs for the test case.
• Call: Executes the object or method under test
• Assertion: Verifies actual results against the expected, passing if true, failing if false.

Unit Test Effectiveness

• Unit test effectiveness measures test cases, confirming component functions and identification of defects.
• Test cases confirm the normal expected component behavior
• It should find defects if they exist
• Unit test cases should include abnormal usage situations to ensure gracefully handling and robust error handling.

Unit Testing Types

• Testing types include normal operations and edge case error handling

Testing Strategies

• Common strategies include Partition and Guideline based tests

Partition Testing

• In partition testing Inputs are divided into groups with common characteristics for specific tests.
• Test cases are selected from each group to ensure all inputs are processed.

Guideline-Based Testing

• Testing guidelines are used to select test cases.
• Using insights from previous experiences find common programming errors, this improves the effectiveness of tests.

Partition testing details

• Inputs and outputs are grouped into equivalence partitions and treated similarly by the program
• Equivalence partitions are domains where the program behaves consistently for all members
• Select test cases from each, and this will confirm all test can be handled correctly within class.

Sequence Testing Guidelines

• Types cases include single-value, varied size and element access

Sequences Testing Scenarios

• Single-Value : Ensure it can handle and process the smallest size sequence.
• Varied Size : Test handling varying length sequences.
• Element Access : Test the first, middle, and last tests.
• Zero-Length : Confirm proper handling of empty sequences without errors.

General Testing Guidelines

• General testing guidelines will help cover a range of scenarios and edge cases.
• These are designed to cover systems weaknesses and robustness.
• Types can include messages, buffers and outputs

Generate All Errors Messages

• Trigger every possible error message to verify the system correctly identifies and handles errors.

Handle Input Buffer Overflow

• Design input buffers that overflow to assess error handling with large inputs.

Repeated Inputs

• Repeatedly use the same input or series to ensure consistent handling and check for issues.

Invalid Outputs

• Force invalid output generation to test the system’s ability to handle and recover them.

Extreme Computation

• Generate both extreme results to confirm extreme values without overflow or underflow issues.