Software testing principles

• Testing is a process consisting of lifecycle activities, both static and dynamic.
• Testing is concerned with the planning, preparation, and evaluation of a system or component, related work products, to determine if they satisfy specified requirements.
• Testing shows if they are fit for purpose and to detect any defects.

Objectives of Testing

• Testing prevents defects through evaluating work products.
• Requirements and user stories are examples of work products
• Testing builds confidence in the test object's quality.
• Testing verifies if all specified requirements have been fulfilled.
• Testing finds defects and failures, reducing the risk of inadequate software quality.
• Testing checks if the test object is complete.
• Testing checks if the test objects validate and work as stakeholders expect.
• Testing complies with legal, contractual, and regulatory requirements and standards.
• Testing verifies the test object's compliance with requirements and standards.
• Testing provides sufficient information to stakeholders to allow them to make informed decisions about the test object's quality level.
• Testing is dependent on the context of the component/system being tested, the test level, and the software development lifecycle model.

Testing vs Debugging

• Testing shows failures caused by defects in the software and is done by testers.
• Debugging is a development activity that finds, analyzes, and fixes defects, which is done by developers.

Why is Testing Necessary

• Testing's contributions lead to success.
• Testing identifies requirements defects through requirements review.
• Testing reduces the risk of incorrect or untestable features being developed through early testing.
• Testing works closely with developers to understand the code and how to test it.
• Testers reduce the risk of defects within the code and the tests.
• Quality Assurance focuses on confirming software corresponds to spec.
• Quality Assurance focuses on quality control and meeting requirements.
• Quality Assurance has a process-oriented orientation.
• Quality Assurance is preventative.
• The goal of QA is to assure quality.
• Testing explores a system to find defects, inspect the system and find bugs.
• Testing has a product-oriented orientation.
• Testing is corrective, and the goal is to control the quality.

Errors, Defects and Failures

• Errors/Mistakes are human actions that produce incorrect results.
• Faults/Defects/Bugs are flaws in a component or system that can cause it to fail.
• Failures deviate the component or system from its expected delivery, service, or result.

Root Cause

• Root cause analysis involves actions/conditions that cause defects.
• It can lead to improved processes that can prevent future defects.
• Causes are identified to reduce the future occurrences of similar defects.

Error Causes

• Time constraints can cause errors
• Human fallibility can cause errors
• Inexperienced or insufficiently skilled project participants can cause errors
• There can be miscommunication between project designers that can cause errors
• Complexity of code can cause errors
• There can be misunderstanding about intra-system and inter-system interfaces
• New or unfamiliar technologies can cause errors

Seven Testing Principles

• Testing shows presence of Defects: Testing demonstrates the existence of defects but cannot prove their absence; testing helps to find undiscovered defects.
• Exhaustive testing is impossible, so smarter testing should be adopted.
• Early Testing: Finding defects early saves money.
• Testing is Context Dependent: Different sites are tested differently.
• Defect Clustering: Defects may cluster in small pieces of code.
• Pesticide Paradox: Repeating test cases will not find new bugs, review and modify them instead.
• Absence-of-errors fallacy: Finding and fixing bugs doesn't help if the solution doesn't meet user requirements.

Test Process in Context

• Factors influencing the test process:
• Software development lifecycle model and project methodologies in use
• Product and project risks
• Test levels/types considered
• Business domain.
• Operational constraints, including budgets, timescales, and complexity
• Contractual and regulatory requirements.
• Organizational policies and practices
• Required internal/external standards.

Test Activities and Tasks

• Activities in test plan:
• Test planning
• Test monitoring and control
• Test analysis
• Test design
• Test implementation
• Test execution
• Test completion

Test Planning

• Test planning defines objectives of testing and the approach for meeting test objectives.
• It imposes objectives within constraints of the context, and it may be revisited based on feedback.

Test Monitoring and Control

• Test monitoring compares actual and planned progress using metrics defined in the test plan.
• Test control takes actions necessary to meet plan objectives.
• Exit criteria sets conditions for officially completing a defined task.

Test Analysis

• Analysis involves the test basis appropriateness to the level being considered, evaluating the basis/test items to identifying defects of various types.
• This process includes identifying features/sets of features to be tested and defining/prioritizing test conditions for each feature based on analysis.
• It captures bi-directional traceability between each element of the test basis and associated test conditions.
• Test basis is the body of knowledge for test analysis (example: SRS, design document, use cases)
• Test item is part of a test object used in the test process (example: login process, report generation).
• Test condition is a testable aspect of a component or system as a basis for testing (example: error handling test cases).

Test Basis Defects

• Types of test basis defects include ambiguity, omissions, inconsistencies, inaccuracies and contradictions.

Test Design

• Test design involves prioritizing and designing test cases/sets of test cases, and it identifies test data to support test conditions/test cases.
• It designs the test environment and identifies the needed infrastructure and tools while capturing bi-directional traceability.
• Test case consists of preconditions, inputs, actions (applicable), expected results and post-conditions, developed on test conditions.
• Test data is needed for execution.
• The test environment contains hardware, instrumentation, simulators, software tools, and support for testing.

Test Implementation

• The process develops and prioritizes test procedures and may create automated test scripts or create test suites from procedures.
• The process arranges test suites within a test execution schedule, building the test environment and verifying that everything has been set up correctly.
• It prepares test data and ensures that it is properly loaded, and updates bidirectional traceability.
• A test procedure is a series of test cases in execution order that may require actions.
• A test suite is a set of test scripts/procedures.
• Test execution refers to running a test on a component/system, which creates actual results.
• A test harness is a test environment composed of drivers and stubs.

Test Execution

• Test execution records the IDs and versions of the test item(s) or test object and tool(s).
• It executes tests (either manually or automatically), and it compares actual results with expected results while analyzing anomalies.
• The root cause for failures may occur because of code defects, but false positives also may occur.
• Defects identified are reported, and the outcome is logged which is either pass, fail, or blocked.
• Test activities are repeated, and bi-directional traceability updating is verified.

Test Completion

• Test completion confirms closed defect reports and backlog items for unresolved defects.
• A test summary report should be communicated to stakeholders.
• The test environment is finalized and archived and lessons from iterations/releases are analyzed.
• The testware is handed over, and tests processed are improved.

Traceability

• Supporting good traceability:
• Analyzing the impact of changes
• Making testing auditable
• Meeting IT governance criteria
• Improving the understandability of testing and reports
• Relating the technical aspects that stakeholders can understand
• Providing information to assess quality, capability, and progress against business goals

Psychology of testing

• The psychology of testing explains the difference between developer and tester roles.
• Testers point out problems with software, while developers like to think that their software is perfect.

Mindsets

• Testers need curiosity, pessimism, a critical eye, attention to detail, and motivation.
• Developers favor confirmation bias and designing/building solutions.

Improving Communication

• Ways to improve communication between a tester and a developer:
• Start with collaboration instead of battles and focus on communication, products, and feelings.
• Always confirm messages have been understood, and focus on the common goal of better quality systems.