Introduction to Software Testing

Introduction to Software Testing

• Software testing aims to show a program functions as intended and identifies defects before deployment.
• Testing involves executing a program with artificial data and checking results for errors, anomalies, or information about non-functional attributes.
• Testing reveals errors but cannot confirm their absence.

Topics Covered

• What is software testing?
• Verification vs. validation
• Fault (bug)
• Software inspection and testing
• Test cases
• Test plans
• Stages of testing
• General testing guidelines
• Requirement-based testing

Program Testing Goals

• Demonstrate to developers and customers that the software satisfies its requirements.
• Custom software: At least one test per requirement in the requirements document.
• Generic software: Tests for all features and their combinations.
• Discover situations where software is incorrect, undesirable, or does not comply with specifications.
• Defect testing: Identify undesirable behavior such as crashes, incorrect system interactions, and data corruption.

Validation and Defect Testing

• Validation testing: Assesses if the system performs correctly according to expected use cases.
• Defect testing: Creates test cases to uncover defects; these cases may be unconventional and do not need to reflect normal usage.

Testing Process Goals

• Validation testing: Demonstrates that the software satisfies its specifications and operates as intended.
• Defect testing: Finds faults or defects in the software where behavior is incorrect or does not match specifications. A successful test reveals a defect in the system through incorrect operation.

Input-Output Model of Program Testing

• Inputs: Artificial data used to drive the system.
• System: The program being tested.
• Outputs: Results showing the presence of defects.

Verification vs. Validation

• Verification: Focuses on "building the product right," ensuring that the software conforms to its specifications.
• Validation: Addresses "building the right product," confirming the software meets user needs.

V&V Confidence

• Aim: To establish confidence that the system is fit for its intended purpose.
• Depends on:
  • System purpose.
  • User expectations.
  • Marketing environment (e.g., time to market).
• Software purpose: Confidence level depends on how crucial the software is to the organization.
• User expectations: Users may have varying levels of expectation for different types of software.
• Marketing environment: Early market release may be more important than finding program defects.

What is a Fault?

• A fault (bug) is an error in software causing unexpected or incorrect behavior.
• Types of faults:
  • Syntax faults (code syntax errors).
  • Logic faults (errors in program logic).
  • Runtime faults (errors during program execution).

Inspections and Testing

• Software inspections: Analyze static system representations (requirements, design, code) to find issues. Supplement with tool-based analysis.
• Inspections don't run the system, enabling them to be applied at early stages before implementation.

Software Testing and Inspection

• Software Testing: Concerned with exercising and observing product behavior by executing the system. Observing its output with provided test data.
• Inspection: Software inspections analyze static representations of the software (requirements, design, configuration data, schemas, UML design, program, etc.)
• Both methods are complementary and both are useful during the verification and validation phase.

Advantages of Inspections

• Errors during testing can hide other defects. Static inspections overcome that problem.
• Incomplete systems can be inspected without specialized tools.
• They can consider broader program attributes such as compliance, portability, and maintainability.

Stages of Testing

• Development testing: System tested as components are developed to find bugs and defects.
  • Unit testing: Testing individual components/modules.
  • Integration testing: Verifying how components work together.
  • System testing: Checking the entire system's behavior.
• Release testing: Separate team evaluates a complete system version before release to users to ensure specified features, functionality, and performance requirements are met.
• User testing: Users or potential users test the system in real-world scenarios.
  • Alpha testing: Preliminary user testing at the developer's site.
  • Beta testing: User testing with a nearly finished product in users' environments.
  • Acceptance testing: Users confirm the software meets their needs before deployment.

General Testing Guidelines

• Create inputs forcing error messages.
• Design inputs to cause input buffers to overflow.
• Repeatedly test with the same input or input series.
• Force invalid outputs.
• Force computation results to be too large or too small.

Testing Policies

• Exhaustive system testing is impossible, so policies defining required system test coverage are developed.
• Specific cases that must be handled: All system functions accessible through menus, menu function combinations, checking for both valid and invalid inputs.

Requirements-Based Testing

• Analyze requirements to generate test cases.
• Ensures the system functions as expected.
• Helps identify missing or incomplete requirements.
• Steps: Identify requirements from specifications, design tests for each requirement, and verify that the software behaviors as expected.

Key Points

• Testing can only show the presence of errors, not their absence.
• Development testing is the responsibility of the software development team.
• Separate teams often handle release and user testing.
• Use experience and guidelines to create effective test cases.
• Develop automated tests to be run whenever code changes are made.