Software Testing Fundamentals

• Software testing assesses software quality to reduce failure risks.
• It involves defect discovery and work product quality evaluation.
• Test objects are work products being tested.
• Testing includes activities beyond just running tests and must align with the software development lifecycle (SDLC).
• Testing involves verification (meeting requirements) and validation (meeting user needs).
• Dynamic testing executes software, while static testing does not.
• Static testing includes reviews and static analysis.
• Dynamic testing uses techniques and approaches to derive test cases.
• Testing requires planning, management, estimation, monitoring, and control.
• Testers need specialized knowledge, analytical skills, critical thinking, and systems thinking.
• The ISO/IEC/IEEE 29119-1 standard provides information about software testing concepts.

Test Objectives

• Evaluating work products.
• Causing failures and finding defects.
• Ensuring required coverage of a test object.
• Reducing the risk level of inadequate software quality.
• Verifying specified requirements.
• Verifying compliance with contractual, legal, and regulatory requirements.
• Providing stakeholders with information for informed decisions.
• Building confidence in the quality of the test object.
• Validating that the test object is complete and works as expected.
• Test objectives depend on context (work product, test level, risks, SDLC, business context).

Testing and Debugging

• Testing and debugging are distinct activities.
• Testing finds defects (static) or triggers failures (dynamic).
• Debugging involves finding, analyzing, and eliminating the causes of failures triggered by dynamic testing.
• The debugging process includes failure reproduction, diagnosis, and fixing the defect.
• Confirmation testing verifies the fix, preferably by the original tester.
• Regression testing checks for new failures caused by the fix.
• When static testing finds a defect, debugging focuses on removing it immediately.

Why is Testing Necessary?

• Testing is a form of quality control that helps achieve test objectives within constraints.
• All stakeholders can use testing skills to contribute to project success.
• Testing identifies defects in software components, systems, and documentation.

Testing’s Contributions to Success

• Testing provides a cost-effective means of detecting defects that can be removed.
• Testing helps evaluate a test object quality at different SDLC phases.
• The measures are used for project management and decisions.
• Testing represents users' needs, which ensures that their understanding is considered throughout the SDLC.
• Testing is required to meet contractual, legal, or regulatory requirements.

Testing and Quality Assurance (QA)

• Testing and QA are not the same.
• Testing is product-oriented and corrective, focusing on achieving quality levels. It is a type of quality control.
• QA is process-oriented and preventive, focusing on process implementation and improvement.
• QA applies to both development and testing, making it everyone's responsibility.
• Testing fixes defects, QA provides feedback on development and test process performance.

Errors, Defects, Failures, and Root Causes

• Humans make errors (mistakes) that lead to defects (faults, bugs), which may cause failures.
• Defects can be found in documentation, source code, or supporting work products.
• Defects early in the SDLC can lead to defective work products later.
• A failure occurs when a defect in code is executed and causes the system to malfunction.
• Failures stem from errors, defects, or environmental conditions.
• A root cause is a fundamental reason for a problem.
• Root cause analysis identifies root causes to prevent future similar issues.

Testing Principles

• These offer general guidelines applicable to all testing.

• Testing shows the presence, not the absence of defects.

• Testing can show that defects are present but it cannot prove their absence.

• Testing reduces the probability of undiscovered defects and does not guarantee the test object correctness.

• Exhaustive testing is impossible.

• Testing everything is not feasible except in trivial cases.

• Test techniques, test case prioritization, and risk-based testing should be used to focus test efforts.

• Early testing saves time and money.

• Defects removed early do not cause subsequent defects.

• Static and dynamic testing should be started as early as possible.

• Defects cluster together.

• Most defects are typically contained within a small number of system components.

• This is an illustration of the Pareto principle.

• Predicted and actual defect clusters are important input for risk-based testing.

• Tests wear out.

• Repeated tests become ineffective at finding new defects.

• Existing tests and data might need to be modified and new tests written.

• Repeating the same tests can have a beneficial outcome, e.g., in automated regression testing.

• Testing is context dependent.

• There is no one-size-fits-all approach to testing.

• Testing is done differently based on different contexts.

• Absence-of-defects fallacy.

• It is a misconception to think software verification ensures a system's success.

• Thorough testing and fixing defects may still result in a system that does not meet user needs.

• Validation should also be carried out.

Test Activities and Tasks

• Activities form a test process.

• The test process can be tailored to a given situation based on multiple factors.

• Test planning: Defining test objectives and selecting the best approach within the constraints.

• Test monitoring and test control: Checking test activities and taking actions needed to meet test objectives.

• Test analysis: Analyzing test basis, identifying testable features, defining and prioritizing test conditions, evaluating the test basis, and use of test techniques. Test analysis answers "what to test?".

• Test design: Elaborating test conditions into test cases, identifying coverage items, defining test data requirements, designing the test environment, and use of test techniques. Test design answers "how to test?".

• Test implementation: Creating testware, organizing test cases, creating test scripts, and building the test environment.

• Test execution: Running tests, comparing results, logging results, and analyzing anomalies.

• Test completion: Creating change requests for unresolved defects, archiving testware, shutting down the test environment, analyzing lessons learned, creating a test completion report, and communicating with stakeholders.

Test Process in Context

• Testing is related to stakeholders and the fulfillment of business needs.
• Stakeholders includes needs, expectations, requirements, willingness to cooperate, etc.
• Team members includes skills, knowledge, level of experience, availability, training needs, etc.
• Business domain includes criticality of the test object, identified risks, market needs, specific legal regulations, etc.
• Technical factors includes the type of software, product architecture, technology used, etc.
• Project constraints include scope, time, budget, resources, etc.
• Organizational factors includes organizational structure, existing policies, practices used, etc.
• Software development lifecycle includes engineering practices, development methods, etc.
• Tools includes availability, usability, compliance, etc.
• These factors impact test strategy, test techniques, automation, coverage, detail, and reporting.