Unit Testing: Static and Dynamic Approaches

Concept of Unit Testing

• Unit/module testing tests program units in isolation
• Examples are functions, procedures, methods, and classes
• It is performed by the programmer familiar with the code
• Static unit testing is non-execution-based
• Examines code for all possible behaviors during run time and validates each unit's requirements by reviewing code
• C++ example: using abort() or exit() instructions
• Dynamic Unit Testing is execution-based
• A program unit is executed, and its outcomes are observed to evaluate system quality
• Static unit testing is not an alternative to dynamic unit testing. They are complementary
• Partial dynamic unit testing is often done concurrently with static unit testing
• Static unit testing is recommended before dynamic unit testing

Static Unit Testing

• Code is reviewed using techniques such as inspection and walkthroughs
• Inspection is a step-by-step peer review with pre-determined criteria
• Walkthrough is a review where the author leads the team through a manual or simulated execution using pre-defined scenarios
• Examine source code in detail systematically
• Code review objective is to review, not evaluate the author
• Code review needs planning and management
• Key to success: divide and conquer by inspecting small parts in isolation, ensuring nothing is missed
• The correctness of examined parts implies the correctness of the whole module
• Main steps: Readiness, Preparation, Examination, Re-work, Validation, Exit

Static Unit Testing (Code Review) - Readiness

• A unit is ready if it satisfies the completeness, minimal functionality, readability, complexity, and requirements criteria
• Completeness requires all code to be available
• Minimal functionality requires that the code is compiled and linked to ensure its basic functionalities
• Complexity refers to the number of conditions, inputs, and outputs
• Team roles include moderator, author, presenter, record keeper, reviewers, and observers

Static Unit Testing (Code Review) - Preparation

• Each reviewer reviews the work package and develops a list of questions
• Examples: Does the code meet design specifications?; Does the module solve the problem correctly?; Is there duplicate code?
• Potential Change Requests (CR) are raised as change requests rather than defect reports
• Suggested improvement opportunities may be suggested by the reviewers

Static Unit Testing (Code Review) - Examination

• The author presents the code
• The presenter reads the code
• The record keeper documents the CR
• The Moderator ensures the review is on track
• Re-work involves the record keeper summarizing the meeting, including the list of CRs, of improvements, and meeting minutes
• The author then works on the CRs to fix the issues
• Validation means CRs are independently validated
• Exit results in a summary report of the meeting minutes
• A Change Request (CR) includes a description of the issue, priority level, assigned person, and deadline

Static Unit Testing - Metrics

• Lines of code reviewed per hour (LOC/Hr)
• Change Requests generated per hour (CRs/Hr)
• Change Requests generated per thousand lines of code (CRs/KLOC)
• Total hours spent on the code review process (KHr)

Static Unit Testing - Applications & Documentation

• Code review methodology applies to other documents
• Five system document types include Requirements, Functional Specification, High-level Design, Low-level Design, and Code
• Installation, User, and Troubleshooting guides developed by the technical documentation group

How to Prevent Defects

• Build instrumentation code and ensure clean code
• Use standard control to detect error conditions (e.g., division by zero)
• Include loop counters and define variables for decision logic branches
• Ensure all return values are handled
• Handle counter data fields and buffer overflow/underflow appropriately
• Validate input data
• Provide error messages in help texts from a common source
• Use assertions to detect impossible conditions
• Leave assertions in code for performance, document unclear ones
• After major computation reverse to compute inputs from results

Dynamic Unit Testing

• Unit environment is emulated and tested in isolation
• The calling unit is the test driver, which invokes the unit under test (UUT), provides input, and reports results
• Emulated units called by the UUT are stubs, which are dummy programs
• Test drivers and stubs form scaffolding
• The low-level design document provides test data selection guidance

Dynamic Unit Testing - Common Techniques

• Control flow testing involves:
• Drawing a control flow graph (CFG) from a program unit
• Select a few control flow testing criteria
• Identifying a path in the CFG to satisfy the selection criteria
• Deriving the path predicate expression from the selection paths
• Solving the path predicate expression for a path to generate test data
• Data flow testing follows a similar procedure to control flow testing but using data flow
• Domain testing: define domain errors, then select test data
• Functional program testing: define input/output domains to compute input values for expected output values

Mutation Testing

• Mutation testing measures the quality of test cases and adds benefits, but should supplement standard techniques
• A mutation introduces a slight syntactical change
• Mutant is a modified program
• A mutant is killed/dead if a test case execution causes it to fail
• A mutant is stubborn if the existing test cases don't kill it
• A mutant is equivalent if it always produces the same output
• A mutation score is the percentage of non-equivalent mutants killed
• The test suite is mutation-adequate if the score is 100%

Mutation Analysis

• A two-step process
• Determine adequacy of an existing test suite to distinguish the program from its mutants
• Mutants not killed are stubborn
• Add test cases to kill stubborn mutants, repeating until the desired score is reached

Mutation Testing Steps

• Begin with a program, and correct tests
• Run tests against the program and fix failed tests
• Create mutants each differing simply from the program
• Execute the tests against each mutant
• If any mutant results fail, it's considered incorrect
• If no mutants fail their behaviors cannot be distinguished
• Calculate a mutation score equal to D/(N −E), where D is the dead mutants, N the total number of mutants, and E the number of equivalent mutants
• If the threshold is too low distinguish all failed tests

Mutation Testing - Underlying Assumptions

• Competent programmer hypothesis: Programmers typically avoid creating random programs
• Coupling effects: Complex faults are coupled to simple faults

Debugging

• Process of determining the cause of failure. It is time consuming and error-prone
• Involves evaluation, intuition, and luck
• Isolates and determines the specific cause
• Approaches to debugging: brute force, cause elimination, backtracking
• Cause elimination includes induction and deduction

Unit Testing in eXtreme Programming (XP)

• Pick a story, write a failing test case, implement code to pass the test, execute all tests, rework code until all tests pass, repeat until the story is completed
• Three laws of Test Driven Development (TDD)
• Write production code after failing tests
• Write less unit test
• One person develops code and the other inspects it
• XP code is developed by two programmers side by side
• One develops code tactically, the other inspects it methodically