Software Testing Overview

Software Testing Lecture Notes

• Software testing aims to demonstrate a program's functionality and identify defects before deployment.
• Testing involves executing the program with sample data and checking results for errors, anomalies, or issues with non-functional attributes.
• Tests reveal the presence of errors, but not their absence.
• Testing is a part of a broader verification and validation process, also including static validation techniques.

Program Testing Goals

• Demonstrate to the developer and customer that the software meets its requirements.
  • Custom software requires at least one test for each requirement.
  • Generic software requires tests for all features and their combinations.
• Discover situations where the software's behaviour is incorrect, undesirable, or doesn't conform to its specification.
  • Defect testing concentrates on finding undesirable system behaviour (e.g., crashes, incorrect computations, data corruption).

Validation and Defect Testing

• Validation testing ensures the system performs correctly for its expected use cases.
• Defect testing focuses on identifying defects, potentially using obscure test cases, not necessarily reflecting normal use.

Testing Process Goals

• Validation testing ensures the software meets its requirements and functions as intended.
• Defect testing aims to find faults or defects in the software where its behaviour is not compliant with the specifications.

Input-Output Model of Program Testing

• Input data is processed by the system, which produces output results.
• Anomalous behavior in the system is identified via outputs, revealing defects.

Verification vs Validation

• Verification asks: "Are we building the product right?". The software must comply with the specification.
• Validation asks: "Are we building the right product?". The software must do what the user needs.

V&V Confidence

• The aim of verification and validation (V&V) is to establish confidence that the system is fit for purpose.
• Confidence depends on the system's purpose, user expectations, and the marketing environment.
  • Critical software for an organization requires higher confidence levels.
  • Users may have low expectations for certain software types.
  • Faster market entry may override thorough defect detection.

Inspections and Testing

• Software inspections analyze static system representations (e.g., requirements, design, code) to uncover problems.
• Inspections may use tools to help identify defects.
• Software testing evaluates a system by executing it with test data to observe its behavior.

Advantages of Inspections

• Testing can mask some errors.
• Static analysis avoids the need to handle intricate interactions amongst various error points.
• Incomplete versions can be inspected without extra costs.
• Inspections can identify quality attributes like compliance, portability, and maintainability.

Inspections and Testing - Complementary

• Inspections and testing supplement each other, not oppose each other.
• Both should be part of the V&V process.
• Inspections check conformance to specifications, but not necessarily customer needs.
• Inspections do not cover non-functional characteristics (like performance, or usability).

Software Testing Process Model

• A systematic model defines and describes the steps for implementing testing on a system.
  • Design tests.
  • Prepare tests cases data.
  • Execute the tests against the system.
  • Compare results.
  • Prepare test reports

Stages of Testing

• Development testing: Involves testing during development to find and fix bugs.
• Release testing: A team independent from development tests a complete system (before release).
• User testing: Users test the system in a real, end-user environment.

Development Testing

• Primarily addresses the specific units and objects within a system, ensuring they function at the unit level.
  • Unit testing: Individual program units or object classes are tested; focuses on object functionality.
  • Component testing: Individual units are integrated to create composite components; focuses on component interfaces.
  • System testing: All components are integrated ensuring interactions work properly.

Unit Testing

• Tests individual components in isolation.
• A defect-detection process.
• Units include functions, methods within an object, whole object classes, and composite components with defined interfaces to access their functionality.

Object Class Testing

• Comprehensive test coverage needs testing all object operations, attributes and states.
• Inheritance complexities can impact the ability to localize information necessary for thorough test design.

Weather Station Object Interface

• Illustrates the methods and attributes of a weather station object (e.g., reportWeather, remoteControl).

Weather Station Testing

• Test cases should cover actions like reportWeather, calibrate, test, start-up, and shutdown.
• State models can help identify transition sequences and the corresponding events.

Automated Testing

• Unit testing should be automated to streamline the process.
• Utilize test automation frameworks (e.g., JUnit) to automate writing and running tests.
• Frameworks often provide generic test classes that can be extended to create specific test cases, and often display results graphically.

Automated Test Components

• Setup: Initializes the system with inputs and expected outputs for a test case.
• Call: Executes the unit (e.g., object or method) under test.
• Assertion: Compares the result with the expected output.

Unit Test Effectiveness

• Test cases should demonstrate expected component behavior.
• Tests should expose defects by revealing errors in component behaviour.
  • Tests reflecting normal operations and those based on experience of common problems.

Testing Strategies

• Partition testing: Groups inputs for similar processing.
• Guideline-based testing: Uses testing guidelines to select specific test cases and to identify typical programmer errors.

Partition Testing

• Groups data with similar behavior or attributes together.
  • The data groups are referred to as equivalence partitions.

Equivalence Partitioning

• Divides input/output values into partitions, where similar behavior for each member is anticipated.
• Tests need to be chosen from each partition.

Testing Guidelines (sequences)

• Test with sequences of varying size.
• Test with sequences with specific elements that need access (like start, middle, end).
• Test with empty (zero-length) sequences.

General Testing Guidelines

• Force various error messages.
• Cause input buffers to overflow.
• Repeat the same or similar input many times.
• Force generation of invalid outputs.
• Generate computational results far beyond acceptable ranges.

Key Points

• Testing demonstrates error presence but doesn't prove absence of all errors.
• Responsibility of testing a system belongs to a group/team separate to development.
• Testing includes unit testing, component testing, and system testing.

Release Testing

• A testing process for a system released outside the development team.
• A critical aim is to prove sufficient quality for actual use.
• Usually a black-box testing process (derived from system specifications).

Release testing and System testing

• Release testing is a form of system testing.
• An independent testing team is often employed for release testing.
• Distinct from system testing, release testing emphasizes validation to ensure the system's readiness for external clients.

Requirements Based Testing

• Software testing aligned with specific requirements.
• Examples:
  • Allergy warnings in medical software systems.

Requirements Tests

• Specific tests related to each specific requirement.
  • Testing examples with patient allergy information.

Features Tested by Scenarios

• A set of planned scenarios for actual use.
• Testing for specific functionality.

A Usage Scenario

• Example showing practical use and testing aspects across system components.
  • Practical example covering user actions and system responses for various scenarios.

Performance Testing

• Assessing the performance and reliability of the system.
• Tests should reflect expected system usage profiles.

Stress Testing

• System overload to measure its ability to respond.
  • Measures the system's behavior during periods of high load.

User Testing

• Involves end-users and customers in the testing process.
• Crucial stage even when other testing types are complete.
• Reflects the variability in user environments with real-world uses and issues.

Types of User Testing

• Alpha testing: Users and developers collaborate at the developer's site.
• Beta testing: Users receive a release for testing outside a development environment.
• Acceptance testing: Customers decide if the system ready for deployment.

The Acceptance Testing Process

• Steps of acceptance testing (e.g., defining criteria, tests, results, outcome).

Stages in the Acceptance Testing Process

• Listing the steps involved.

Agile Methods and Acceptance Testing

• User/customer involvement in development decisions.
• Tests are defined and integrated throughout the agile process.

Key Points (Summary)

• Identifying the common error patterns.
• Using effective testing techniques.
• Applying test automation whenever possible.
• Test-driven development: Tests written first; code following.
• Scenario testing, defining use cases.
• Acceptance testing: Validating if the system meets the client's expectations.

This quiz covers key concepts in software testing, including the goals and methods used to validate software functionality and identify defects. It discusses the importance of executing tests with sample data and the role of defect testing in ensuring software quality. Test your understanding of the verification and validation processes involved in software development.