Software Testing Fundamentals

Introduction to Software Testing

• Testing validates a program through a planned process, beginning early in development.
• Testing is considered a positive activity that seeks to show that the program meets its design specification
• The purpose of testing is to find errors and bugs.
• Debugging removes errors, and it is a negative, reactive activity that cannot be planned.

The Inevitability of Bugs

• Research indicates even good programs contain 1-3 bugs per hundred statements.
• Programmers should adjust their view of testing and debugging and embrace the process.
• Debugging can be more challenging than the initial coding exercise.

Objectives of Testing

• Testing affirms product quality and helps eliminate errors from previous development stages.
• Testing also demonstrates the existence of specified functionality.
• Testing validates software as a solution to the original problem.
• Operational reliability of the system is estimated through testing.

Verification vs. Validation

• Verification determines, "Are we building the product right?"
• Validation determines, "Are we building the right product?"

Stages of Testing

• Unit testing involves testing individual modules as standalone entities.
• In object-oriented programming, a "unit" can be a class or related classes.
• Integration testing involves testing combined individual components that create subsystems.
• Components may work separately but fail when combined due to interface mismatches, for example.
• Validation testing involves testing the whole system in a realistic setting.
• Alpha tests are conducted at the developer's site by a customer in a controlled environment.
• Beta tests occur at one or more customer sites by end-users without the developers.
• Beta tests are for customers to record all problems and report to the developer
• Acceptance tests are formal tests to determine if a user, customer, or authorized entity accepts the system or component.
• System testing addresses issues concerning the whole system and its relationships with other systems.
• Performance, security, and configuration are issues tested during system testing.
• Performance testing or load testing, is important for network or web-based applications with many simultaneous users.

Testing Strategies

• Testing starts at the module level and expands outward through the system.
• Different testing techniques are used for each stage of the testing process.
• Testing can be done by developers and sometimes by independent teams.
• Debugging is required at every stage of the testing process.

Unit Testing Issues

• Test cases should likely identify errors.
• Test case design should give the widest possible coverage for exercising the software unit.
• Black box functional testing and white box structural testing are two key techniques.

Black Box Testing

• Software elements is viewed as a "black box”.
• The concerns of the tester does not include internal operations.
• Test cases are created from the functional requirement of the unit.
• Confirms that when inputs are accepted, the corresponding outputs are correctly produced.

White Box Testing

• White box testing entails detail to test cases construction
• Actual logic paths through the code are tested through white box testing.
• Specific sets of conditions and loops are tested.
• The program's status can be examined during white box testing.

TDD (Test-Driven Development)

• Unit tests in TDD should mainly use black box testing, since tests are created before code.
• Developing tests based on specified functionality that is to be implemented.
• Truly not black box, since developers may have implementation knowledge.
• Developers may incorporate elements of white box as code and tests evolve in practice.
• BDD (Behavior Driven Development) is clearly black box and demonstrates behavior works correctly.

Black Box Testing and Test-Driven-Development

• In TDD, tests are written first and fail when executed for the first time.
• It acts as scaffolding for code so that small elements can be tested in isolation once coded.
• In this sense, Black Box testing is used to develop test cases.

Defining a "Unit"

• If a class is considered a unit, it can be standalone or related to other classes.
• An Order class may be related to a Product class.
• Testing a method in the Order class may require Product objects to be created.
• This could work in TDD by developing tests and code for Product, then tests/code for Order.
• The "unit" includes Product or Order classes for test purposes.

Mocks and Stubs

• Methods sometimes depend on code outside the unit being tested.
• A mocked stock level service may return a "dummy" value to the Order.
• Unit tests can replace those dependencies, with mocks and stubs, providing a simplified response

Black Box Test Case Design

• Involves selecting typical, extreme, and invalid data values representing the specification.
• It requires a systematic approach rather than a few simple tests.
• Equivalence partitioning and Boundary Value Analysis Techniques are useful.

Equivalence Partitioning

• It partitions the "Input Domain" of a module under test into classes of similar inputs.
• Test data is created for each class.
• A single test case should cover a whole class of input.
• Test case design for equivalent partitioning is founded on an evaluation of equivalence classes and an input condition.
• Depicts a set of valid or invalid states for the input condition, that is considered as the "equivalence class"

Equivalence Classes

• Can be defined based on the following types of input condition:
  • Range: one valid (within range) and two invalid (below and above) equivalence classes
  • Specific value: one valid (equal) and two invalid (below and above) equivalence classes
  • Member of a set: one valid and one invalid class
  • Boolean: one class defined for each condition

Deriving Equivalence Classes

• For each data item, nature of inputs and conditions are identified, such as (range, value, Boolean, and/or set)
• The specified requirements are expressed through a identified condition.
• (e.g.) a three- or four-digit number = expressed as a range of digits between 3 and 5, respectively.
• A Prefix requirement (3-digits, not starting with 0 or 1) = expressed as a value range between 200 and 999

Boundary Value Analysis

• Identifies a great amount of errors that occur at input domain boundaries.
• Boundary Value Analysis is complementary to equivalence partitioning
• Boundary Value Analysis concentrates the selection of test cases at edges of equivalence classes

Guidelines for BVA

• Range: Test cases should be designed with values a & b, and values just below a & above b.
• Various values: Test cases should be produced exercising the maximum & minimum numbers, or the first & last members set.
• Output conditions: Tests which exercise the output domain should be developed that output £0.00 and the maximum amount.
• Internal program: data structures must have prescribed boundaries (arrays)

Selecting Test Data

• The test data for this should cover all equivalence classes and include all of the test cases suggested by BVA.
• Initial test case: tests a set of the classes/values and should lead to the acceptance outcome.
• Followed by: replacing one of the values with another valid value in the same class (6a -> 6b).
• Cases 1-5: valid value sets.
• Cases 6-13: invalid value sets.

Output Equivalence Classes

• It is focused validity of input and testing for correct valid/invalid test cases identification.
• Focus of the traditional approach to black box testing.
• Validity's input tests validity perhaps less important where user UI controls / front end validation.
• Focus also on the output using output equivalence classes.

Writing Test Cases in Java

• Use JUnit to create the test methods.
• Options to use, methods for each test case, and the testing of an object with valid and invalid outputs.
• Adopt clear labeling for meanings of test list output when running a test suite.

Explore the core objectives of software testing, differentiating it from debugging. Understand the importance of verification and validation in ensuring software quality. Learn about unit and integration testing stages.