Black-Box Testing Methods

Black-Box Testing Methods

• Software QA presentation, by Dr. Samer Zein, Birzeit University

Black-Box Testing Intro

• Black-box testing techniques should complement, not substitute, white-box testing.
• Black-box testing ignores internal mechanisms, focusing on outputs from selected inputs, including execution conditions.
• Black-box testing evaluates system compliance with specified functional requirements.

Equivalence Classes Technique

• The focus is to generate a set of test cases
• This set has to be chosen carefully.
• Equivalence class techniques aim to increase testing efficiency and minimize the number of test cases.
• Improves test case selection through efficient use of equivalence class partitioning.
• Equivalent Class: a set of input variable values identically processed to produce the same output.
• Test suites should include both valid and invalid ECs
• The basic rule: there should be one test case for each valid and invalid EC
• This technique minimizes test cases
• The tester defines the EC and its boundaries.
• EC technique is based on requirements' specs, not code
• EC technique is more efficient than automated and random techniques.

Equivalence Partition and Boundary Value Analysis

• Equivalence Partition and Boundary Value Analysis example shows test cases TC 1: -100, TC 2: 0 and TC 3: 100 focuses on behavior and response of the system
• Online video upload website testing example accepts .mp4, .mov, .mkv
• Testing is done by grouping distinct file formats into partitions using equivalence partitioning.
• Select representative files from each partition to cover the website's behavior and functionality
• Requires a test case per file type.
• Boundary Value Analysis (BVA) focuses on testing the boundaries of input values where issues often occur.
• BVA complements Equivalence Partitioning.
• Hidden defects can be uncovered by testing the edges/boundaries of equivalence classes.

Negative Testing

• Ensures an application gracefully handles invalid or unexpected inputs to prevent crashes and incorrect behavior.
• Protects applications from unforeseen issues.
• Some applications have mandatory fields that need to be tested by leaving required fields empty to analyze the application's response.
• Example based on requirements:
• Password needs letters and numbers.
• Users can publish a maximum of 5 microblog posts a day.
• Posts cannot exceed 200 words.
• Only JPEG photos are supported in posts.
• Examples of Test Cases include creating a password with only numbers or letters, attempting to upload non-JPEG files, and writing posts that exceed 200 words.

EC's Boundary Values

• This is additional criteria, with attention needed to test case definitions and test boundaries of each EC.

EC Boundary Values

• Three test cases are needed, middle, lower and upper range
• 3-Value Boundary Value Analysis expands upon upon the 2-value approach by including an additional test case within each boundary - to test invalid and valid values just above those boundaries
• Example: Field accepts values between 1 and 100
• Test Cases: 0, 1, 2, 99, 100, 101
• The general process of identifying EC:
• A) If an input condition specifies a range of values (e.g., "the item count can be from 1 to 999") then: Identify one valid equivalence class (1<item count<999) and two invalid equivalence classes (item count<1 and item count>999).
• B) If an input condition specifies the number of values (e.g., "one through six owners can be listed for the automobile"), then: identify one valid equivalence class and two invalid equivalence classes (no owners and more than six owners).
• C) If an input condition specifies a set of input values with the reason to believe that the program handles each differently, identify a valid equivalence class for each and one invalid equivalence class. (e.g., Type of vehicle BUS, TRUCK, TAXICAB, PASSENGER, or MOTORCYCLE; and TRAILER for the invalid class).
• D) If an input condition specifies a "must-be" situation, identify one valid equivalence class and one invalid equivalence class (e.g., it must be a letter or it is not a letter for the first character of the identifier).

Important Note Regarding Invalid EC

• If the specification states “enter book type (HARDCOVER, SOFTCOVER, or LOOSE) and amount (1–999),”
• The test case, (XYZ, o), expressing two error conditions (invalid book type and amount)
• This probably wont check for the amount, so the program may say "XYZ IS UNKNOWN BOOK TYPE" and not bother to examine the remainder of the input.
• Summary Table of Test Cases Based on EC and BVA (see Table on page 20 of document).
• Equivalence Classes - the Golden Splash Swimming Center ticket price module
• Includes valid and invalid equivalence, representative values and Boundary Values
• Test Cases - the Golden Splash Swimming Center ticket price module includes
• Requires valid and invalid ECS, the day of the week, isitor status, ntry hour and isitor's age and test case results and
• Disadvantages of black box testing include: possible coincidental aggregation of several errors will produce the correct response for a test case, and prevent error detection
• Errors may counteract each other to accidentally produce the correct output
• The absence of control of line coverage, there is no easy way to specify appropriate test cases to improve coverage
• Leads to tests may not execute a substantial proportion of the code lines which are not covered by a set of test cases
• The impossibility of testing the quality of coding and its strict adherence to the coding standards.

Decision Table Testing

• Decision tables help testers understand the effects of different input combinations and software states, which must correctly implement business rules.

State Transition Testing

• An example provides the requirements, user should have a positive account balance in order to transfer money to another user; and the user should not be banned
• A Decision table highlights the condition, steps and expected outcome.
• The number of cases depends on the number of conditions
• The number of test cases = 2^n: e.g. For 2 conditions: = 4 test cases, For 3 conditions: = 8 test cases, For 4 conditions: = 16 test cases.
• State transition testing aims to ensure the system transitions between different states correctly and consistently.
• This is used when features of a system are represented as states that transform into one another.
• Key Test cases include:
• Switching on the device when it's off.
• Switching off the device when it's on.
• Switching on the device when it's already on, resulting in a FAULT state
• Attempting to switch on the device when it's in the FAULT state, which should remain in the FAULT state.
• The goal is ALL transition coverage
• To achieve 100% coverage of all transitions, test cases must exercise all the valid transitions, as well as attempt to execute invalid transitions.
• This ensures comprehensive testing of the state transitions in the system.