Software Testing Fundamentals

Podcast

Play an AI-generated podcast conversation about this lesson

Questions and Answers

What are the three types of flaws or defects identified in software testing?

Mistakes by the software developer, faults in the code, and failures leading to incorrect behavior.

Explain the difference between an algorithmic flaw and a computation/precision flaw.

An algorithmic flaw occurs when input-output mapping is wrong, while a computation/precision flaw occurs when calculated results do not meet expected accuracy.

What are the severity levels of software defects, and which one indicates the most severe issue?

Severity Level 1 indicates the most severe issue, which involves a system crash with expensive recovery times.

What was the cost associated with the software failure related to payment processing?

The exact cost of the failure is unknown, but there was a fine of £56 million. Signup and view all the answers

Name two types of documentation mentioned and their purposes.

User documentation, which describes how to use the system, and design documentation, which is for software developers. Signup and view all the answers

What type of flaw does incomplete or incorrect documentation represent?

It represents a type of flaw related to documentation that can lead to user confusion and operational errors. Signup and view all the answers

What is the total investment in the development of the project up to that point?

$7 billion. Signup and view all the answers

Why is Software Testing considered important in the software development process?

Software Testing is important as it helps in identifying defects early, ensuring quality and reliability in software applications. Signup and view all the answers

What distinguishes white-box testing from black-box testing?

White-box testing involves testing the internal structures or workings of an application, while black-box testing focuses on the outputs based on given inputs without knowledge of the internal implementation. Signup and view all the answers

What has been the trend regarding the importance of Software Testing since the 1970s?

The importance of Software Testing has evolved from a post-development activity to an integral part of the software creation process. Signup and view all the answers

What does the term 'Software Crisis' refer to?

'Software Crisis' refers to the challenges faced during software development in the 1960s and 1970s, including projects going over-budget, over-time, and producing low-quality products. Signup and view all the answers

How does inadequate software testing impact the economy, according to the NIST report from 2002?

Inadequate software testing contributes to approximately $60 billion in losses per year in the US. Signup and view all the answers

What roles does Software Testing play in Quality Assurance?

Software Testing helps ensure the quality of software by identifying defects, validating requirements, and verifying functionality. Signup and view all the answers

What are some of the consequences of reduced software quality?

Reduced software quality can lead to increased failure rates, bad reputation, and potential legal claims against the organization. Signup and view all the answers

Discuss the impact of vague requirements on software development.

Vague requirements can lead to misunderstandings, increased development time, and ultimately lower quality in the software product. Signup and view all the answers

What role does software testing (ST) play in reducing risks during software development?

Software testing minimizes risks by identifying faults early, ensuring software meets specifications, and allowing for scalability while maintaining quality. Signup and view all the answers

List two signs that indicate a software project may fail.

Two signs are cost over-runs and the software not solving the user's problem. Signup and view all the answers

Why is it challenging to collect requirements in software development?

Collecting requirements is hard because it involves accurately understanding user needs and expectations, which can be vague or incomplete. Signup and view all the answers

What is the significance of specifications in the software testing process?

Specifications are crucial as they define software behavior and serve as a reference for testing to ensure the product meets user needs. Signup and view all the answers

In the context of error behavior, what could be a reasonable response if a parameter is missing from a function call?

A reasonable response could be to throw an exception to indicate that a required value is absent. Signup and view all the answers

Describe the criteria for gold customers receiving discounts in the online shop program check.

Gold customers receive a discount when they accumulate 80 points. Signup and view all the answers

What are the two engineering approaches mentioned in software development?

The two engineering approaches are forward engineering and feedback-based engineering. Signup and view all the answers

How does feedback-based engineering contribute to software development?

Feedback-based engineering allows for iterative improvements by incorporating user feedback and identifying issues during the development process. Signup and view all the answers

What defines black-box testing and how does it differ from white-box testing?

Black-box testing examines software functionality without considering implementation, whereas white-box testing analyzes the internal structures of the software. Signup and view all the answers

What is experience-based testing and what role does expert opinion play in it?

Experience-based testing utilizes error guessing and relies on the expert opinion to identify potential faults. Signup and view all the answers

Explain the concept of fault insertion in testing.

Fault insertion involves intentionally introducing faults into code to assess the effectiveness of testing processes in identifying these faults. Signup and view all the answers

How does dynamic testing differ from static testing, and what is its primary focus?

Dynamic testing involves executing code and verifying the outputs, while static testing focuses on reviewing code without execution. Signup and view all the answers

What is the main goal of selecting a minimum set of test data?

To ensure that the software works correctly for all possible inputs. Signup and view all the answers

Describe the purpose of regression testing in the software development lifecycle.

Regression testing ensures that new changes haven’t adversely affected existing functionality by re-running previously completed tests. Signup and view all the answers

What does successful test data ensure in relation to specifications?

All test data in a test set must produce results as defined in the specifications. Signup and view all the answers

Why is exhaustive testing considered reliable and valid?

It selects every value in the program domain, ensuring coverage of all possibilities. Signup and view all the answers

What are the challenges of using exhaustive testing?

It takes a long time to execute every possible test and assess the correctness of results. Signup and view all the answers

What characteristics should good tests possess according to heuristics?

Good tests should have high fault detection probability, no duplications, independence, and cover as much code as possible. Signup and view all the answers

What is the 'Oracle Problem' in software testing?

It refers to the difficulty of determining if the test results are correct. Signup and view all the answers

What issues arise from the Test Data Problem?

Data can be unrepresentative, where some values are over-represented and others are completely missing. Signup and view all the answers

How do heuristics simplify the testing process?

They help reduce the number of tests while maintaining a high likelihood of finding faults. Signup and view all the answers

What is a fault model in the context of test techniques?

A fault model defines the specific faults or issues that a test technique is designed to identify or address. Signup and view all the answers

How does the complexity of fault models change as test techniques evolve?

Initially, test techniques have simple fault models, but they become more sophisticated over time. Signup and view all the answers

What is the primary purpose of software testing?

The primary purpose of software testing is to identify defects and ensure the software functions as intended. Signup and view all the answers

Explain the difference between black-box testing and white-box testing.

Black-box testing focuses on the functionality of the application without looking at internal code, while white-box testing involves knowledge of the internal code structure and logic. Signup and view all the answers

What key distinction exists between faults, mistakes, and failures in software development?

Mistakes are errors made by developers, faults are flaws in code, and failures are when the software does not perform as expected. Signup and view all the answers

Study Notes

Software Testing (CS265) - W1-L2

Software Testing (ST) is a procedure for identifying flaws and defects in software, aimed at creating high-quality, large-scale applications.
The importance of ST is underscored by the significant costs of software failures highlighted by the Ariane-5 rocket failure (1996) and the RBS (2012) incident.
These failures, along with the 1960s/1970s "Software Crisis," highlight software quality issues and the need for ST.
Initial software development primarily focused on fixing bugs, but later included ST as a part of the creation process itself.
Software defects can stem from various sources including mistakes by developers, code faults, and failures due to unanticipated or incorrect behavior under certain conditions.
A significant percentage (around 50%) of software faults can be directly attributed to algorithmic or computational precision issues, according to HP Studies.
Different types of software faults exist, including errors in logic, syntax, and documentation, as well as deficiencies in handling data or capacity.

Learning Objectives

Understand the importance of Software Testing (ST) in software development.
Distinguish between white-box and black-box testing approaches.
Recognize that ST is not a perfect process.

Software Industry History

IBM separated software and hardware in 1969.
Personal computers (PCs) became mainstream in the 1980s.
The World Wide Web emerged in the 1990s, followed by mobile devices in the 2000s.
The software industry's overall value reached (almost) a trillion dollars by 2017.
Significant financial losses (approximately $60 billion annually) are frequently attributed to inadequate software infrastructure.

Importance of Software Testing

Various models and techniques are used for ST.
ST is a crucial aspect of quality assurance and software engineering practices.
The software industry has adopted and adapted various testing strategies in response to the software crisis.
Defining roles, planning, documentation, and process management procedures are all critical components of ST.

Software Testing Definition

Software Testing is a process or strategy to find, assess, and predict defects in software products.
Its aim is to develop reliable, high-quality, large-scale applications.

Software Product Definition

Software products consist of instructions in discrete programs designed to execute and deliver the required functionality.
Data structures, configuration files, and documentation (system, user, and development) clarify functions, use, and design for the software.

Three Types of Defects in Software

Mistakes (developer errors).
Faults (implementation issues).
Failures (undesired behavior).

Types of Software Faults

Algorithmic issues: Incorrect mapping between inputs and outputs.
Computational & precision errors: Calculated results not matching expected accuracy.
Syntax errors: Violations of programming language rules.
Documentation issues: Incompleteness or inaccuracies in documentation.
Capacity/Boundary issues: Failures due to system overload or exceeding capacity/expected parameters.
Degrees of Software Failure (Severity Level): • Level 1 (most severe): System crash, loss of function, no workaround, costly recovery • Level 2: Partial loss of function, data with possible workaround • Level 3: Loss of function or data, manual workaround possible • Level 4 (least severe): Cosmetic issues or minor inconvenience

Software Testing and Quality

Complex code and sometimes vague requirements can contribute to reduced quality and an increased failure rate in software.
Consequences of poor quality include damage to reputation, legal issues.

Why Software Testing

Software testing reduces the risks and costs of software development.
It allows software to be scaled effectively while maintaining quality.
IT diminishes the likelihood of software product failures.
Common signs of software failure include delays, cost overruns, failure to meet user needs, and quality issues in software functionalities

Need for Software Testing

Difficulties in specifying requirements and producing consistent software behaviors contribute to software-related issues.
Software development approaches include forward engineering and a feedback-based approach.

Two Engineering Approaches

Forward Engineering: An ideal approach which involves steps of requirements, specification, design, and implementation followed by reliable development
Feedback Approach: A practical approach using iterative design and implementation with testing and correction after each phase.

Validation in Development Process

Ensure software aligns with users' needs to address validation aspects at different stages of Development

Specifications are Essential

Software specifications provide a critical foundation for testing.
Defining software behavior and expected errors is crucial.
Specifications (including requirements) are expressed in English followed by structured formats (e.g. tables) for testing.

Example: Specification

Discount programs based on customer points and gold membership status.

Example: Return Code

Different return codes (e.g., "FULLPRICE," "DISCOUNT," "ERROR") depending on the points and membership status.

Example: How to Test?

Example test cases for the discount program, including acceptable input values and expected outcomes.

Theory of Software Testing

A theory of ST is needed to minimize the tests required without hindering thoroughness in covering all possible input data, producing results as defined in the specifications, and testing data selection with reliability and validity.

Feasibility of Exhaustive Testing

Exhaustive testing is often impractical due to the extensive time needed to test all possible input values.

Example: Exhaustive Testing

Identification of relevant input parameters and their possible values for exhaustive testing.

Example 2: Code

Identification of the complexities of boundary conditions for bounding conditions related to testing input values.

Test Heuristics

Develop effective testing strategies by employing proven heuristics with various testing types like random testing, white/black box, and other types.

Issues with Random Testing

Oracle problem: Ensuring correctness of results
Test data representation problem: Representative and balanced coverage to avoid issues of some input values being used too often or insufficiently.
Tests completion problem: determining the sufficient number of tests needed for confidence.

Testing Types

Black-box testing: Testing software functions without examining implementation details.
White-box testing: Evaluating software's internals and the internal structures of its components to provide appropriate test cases.
Experience-based testing: Evaluating software using expert insight and estimations to find bugs. (including faults insertion).

When to Finish Testing

Testing should end based on budget parameters, completing planned test cases, or meeting predetermined failure rate criteria.

Static and Dynamic Testing

Dynamic testing: Executing code and verifying its output.
Static testing: Reviewing code without executing it.
Static testing review-based techniques like pair programming, code reviews, and walkthroughs exist for improving the correctness of software processes.
Mathematical techniques for program proving allow formal verification of correctness within the software components.

Testing in the Software Development Process

A structured testing process within the stages of software development including unit, integration, system, sub-system, and acceptance testing stages.

ST Procedure

Procedure of testing processes (ideal vs real) and expected vs actual results.

ST Procedure Example in Testing

Example of an implementation using Java and TestNG for module verification.

ST Activities

Clear description of the steps in software testing, encompassing analysis, identification, verification, implementation, execution, and examination of testing results.

ST Artefacts

Detailed descriptions of the artifacts created throughout the software testing process including coverage items, test cases, test specifications, test code, test results, test procedure documents, and automated testing code.

Fault Models

Describing how fault testing models help facilitate effective testing by matching the input to the fault models and expected outputs.
Examples of fault models (e.g., hand axe, hammer) and how faults are linked to the ideal system.

Summary

A comprehensive overview of software testing, including its purpose, importance, methodologies, and related concepts (mistakes, faults, failures), testing procedures, development process, and different testing models and types.

Studying That Suits You

Use AI to generate personalized quizzes and flashcards to suit your learning preferences.

Description

This quiz covers essential concepts in software testing, including types of flaws, severity levels of defects, and the impact of inadequate testing. Additionally, it explores the distinctions between white-box and black-box testing, as well as historical trends since the 1970s. Test your knowledge and understanding of crucial software testing principles.