Software Testing Fundamentals

Software Testing Basics

• Software testing is the process of evaluating a software solution to ensure it behaves as expected and meets specified requirements.
• It involves executing software under controlled conditions and assessing its behavior to identify defects or bugs.
• A fault, referred to as a bug or defect, is an erroneous software or hardware element in a system that can cause the system to fail.
• Software testing identifies defects, ensures quality, validates requirements, and improves user experience.
• Software testing helps identify defects/bugs, functionality/performance errors, or interface inconsistencies.

The Importance of Testing

• Testing ensures the software meets quality standards and stakeholder requirements, providing reliable and robust products for end-users.
• Testing verifies the software behaves according to specifications and meets user/stakeholder needs.
• Testing enhances the software's overall user experience.
• Testing shows the presence of bugs, not the absence; 100% certainty that software is bug-free is unattainable, but proper testing methodology assists in getting very close.
• The tests should be conducted with a systematic plan instead of random trial and error

Test-Driven Development (TDD)

• Test-Driven Development (TDD) involves writing tests before writing the actual implementation code.
• The first principle of TDD: Every step requires a plan to verify the result meets a goal.
• The second principle of TDD: A software artifact shouldnt be created unless the developer can test it.
• TDD follows the "Red-Green-Refactor" cycle:
  • First, write a failing test case that specifies the desired behavior/functionality of the code (Red).
  • Then, write the minimal code necessary to pass the test (Green).
  • Finally, refactor the code to improve design, readability, and maintainability while keeping tests passing.
• TDD is an iterative process where, after refactoring code, the cycle repeats with modifications to existing tests of additional test cases.

TDD Benefits

• Improved code quality
• Reduced bugs
• Faster feedback loop
• Better documentation

Key Definitions

• A fault/defect/bug is an erroneous hardware/software element that can cause system failure.
• Test case: A particular choice of input data for testing a program, including the expected output/behavior.
• Test: A finite collection of test cases.
• White-box testing: Testing that exploits the internal structure of a program, assuming code is available.
• Black-box testing: Testing that explores the input space of functionality defined by an interface specification.

Why Testing is Hard

• Nontrivial systems cannot be completely tested, and the goal is to find faults as quickly and inexpensively as possible.
• The ideal is to create a single test case that exposes each fault and runs it, but in practice many unsuccessful test cases may be necessary.
• Testing strikes a balance between maximizing test coverage and minimizing economic costs.
• The idea is that correct behavior on "critical" test cases represents the correct behavior on untested parts of the state space.

Logical Organization of Testing

• Testing can be organized a white box or black box.
  • White box is the internal structure, design, and code being tested.
    • With white box, the internal workings of the system are transparent to the tester.
  • Quality testing, the software system is tested from an end-user perspective
    • Whereby The internal workings of the system are hidden from the tester.
• The testing may include unit test, integration testing, system test, function, quality acceptance, installation testing
  • Note the orders are not necessarily strict

Unit Testing

• Unit Testing applies to the smallest part of the system - aka the unit , which is independently tested.
• Unit is sometimes referred to as a fixture.
• To test a unit in isolation mock objects which refers to components external to the unit are created.
  • They are created as a test driver - simulating parts of a system that invoke operations.
  • Unit testing cycle - consists of The unit
• Create the unit, test, driver and test stub(s) and have the test driver invoke an operation on the unit that ensures expected results.

Test Cases

• Acceptance test: A black-box testing approach, which includes:
  • Testing with a valid key of a current tenant on their apartment (pass).
  • Testing with a valid key of a current tenant on someone else's apartment (fail).
  • Testing with an invalid key on any apartment (fail).
  • Testing with the key of a removed tenant on their previous apartment (fail).
  • Testing with the valid key of a newly added tenant on his/her apartment (pass).

Testing Granularity

• Unit Testing: Smallest possible part of a software or system is tested individually and independently.
• Integration Testing: Tests how different components of a system interact with each other and operate together.
• System Testing (end-to-end): The complete system or integrated set of components is tested to verify it satisfies specified requirements and performs as expected as a whole.
• Functional Testing: Tests the system or component which verifies it meets the specified functional requirements.
• Quality Testing: Non-functional testing that evaluates security, performance usability etc
• Acceptance Testing: The customer verifies the requirements have been met.
• Installation Testing: That tests the way the system works and its intended operations and deployment.

Coverage

• Coverage: The degree to which testing has been exercised by tests.
• Test Coverage (AKA Big picture): Focuses on the specifications and checks testing of the input without checking the other parts.
• Code Coverage (Implementation details): Code of coverage Degree to which the program source code has been tested.

Heuristics for High Testing Coverage

• Some tests can more easily pinpoint problems than others, and test cases should be prioritized.
• Equivalence Testing (Mostly Black-box testing)
  • It is a testting method that divides all possiblie inputs into an equivalence group
• Boundary Testing (Mostly Black-box testing)
  • Test the bounds of the inputs
• Control-flow Testing (Mostly White-Box testing)
• State-based Testing (Mostly White-Box testing)

Equivalence

• A black-box testing method
• Divides the input of all possible classes for the system to behave on each same input from the same group
  • Assumes the developer may have made mistakes on classes of the input values.
• The steps are;
  • Partitions what's needed inside the inputs in valid equivalence classes.
  • One or more inputs from each equivalence

Heuristics for finding equivalence classes

• Specified over a range with a Boolean value or not
• Partition that makes two values equivalent

Boundary Testing

• AKA Boundary Value analysis
• It is part of a special case of Equivalence as it touches on the input parameters.
  • Such that Developers fail to see the special cases at the boundary of equivalence classes.
  • Selects inputs from edges of each class or outliers such as zeroes or mins.

Control-Flow Testing

• Organizes testing based on the control flow of the software and its white box.
• The goal to draw a control flow graph to that every statement has a node and transition between edge statements
• In order to measure the code, we use code coverage.

Types of code coverage

• Statement coverage
• Edge coverage
• condition coverage
• Path coverage

State-Based Testing

• Defines a testing method and abstracts the behavior by abstracting object
• It testing behavior by comparing actual states of expected states.
  • Usually is used with Object-Oriented Programming but may be incomplete.
  • The state of an object defines it values of attributes and an object may be unit tested even without the values.

State-Based Testing Components

• State is a constraint on a values of an objects attribues.
• Event is an input given to the a system.
• Action is a result output or operation with the event.
• Transition is a chance of state caused by the event.
• Guard is a condition with an event which depends on the transition.

Test Cases

• It includes all the relevant event:
  • What happens if the User entered a valid key
  • What happens if the User entered an invalid key

State-Based Faults (Control)

• Missing transitions
• Incorrect Transitions
• Inconsistent State transitions
• Loping and infinite Transitions
• Missing/Incorrect Actions Actions
• Traps and paths
• Deadlocks and Livelocks

Integration Testing

• Testing the integration of multiple functions together.
  • Includes Big Bang Integration - which unites multiple systems together.
    • Should not do as this should mean to test the faults only at the whole system. -Includes Bottom testing - lower testing as it relies on top-level execution simulation.
    • Good as a disjoint subsytem tested.
  • Includes Top testing - as the lower levels simulte.
  • Good in terms of functional component

Horizontal Integration Testing

• Assuming a test can be assumed.

Vertical Integration Testing

• Test all components to realis a stories which include the use of inter feedback and outter feedback.

Continous integration

• Software dev has the code integrated into a shared, which trigger build process.

Continous Integration (CI)

• Running test is big part of Continious Integrations
• Every time as built the run of;
  • Unit test as a change component.
  • Regression test on the change component
  • Runs integration test.

Front End Component Testing

• Tests an individual element (UI) or interafce on the website.
  • It unit tests.
  • Uses integration
  • The tool used can snapshot and ensure accessibilty
  • Some even ensure performance test.

Setup Tools For Testing

• Includes Tools related to;
  • Testing Framework
  • Assert Library
  • Hedless Browsers

System Tests

• Quality testing that ensures backwards campatibility
• Followed by attempt to violate test.
• Testing the limits as a measure to see how the system. Acceptance Testing - tests on an development enviroment to get a realistic test in a enviroment. Testing is an art and a science, can hardily guarentee a bug, but tools help make the bugs tractable.