Podcast
Questions and Answers
What is one of the primary benefits of Software Configuration Management (SCM)?
What is one of the primary benefits of Software Configuration Management (SCM)?
Which aspect does SCM help improve in terms of software integrity?
Which aspect does SCM help improve in terms of software integrity?
What type of information does SCM generate about the development process?
What type of information does SCM generate about the development process?
How does SCM contribute to better record keeping?
How does SCM contribute to better record keeping?
Signup and view all the answers
Which of the following is NOT a function provided by the software library in SCM?
Which of the following is NOT a function provided by the software library in SCM?
Signup and view all the answers
What distinguishes software development from manufactured products?
What distinguishes software development from manufactured products?
Signup and view all the answers
Which factor is NOT a significant influence on software quality according to the content?
Which factor is NOT a significant influence on software quality according to the content?
Signup and view all the answers
What is a primary reason that software development processes are organization-specific?
What is a primary reason that software development processes are organization-specific?
Signup and view all the answers
What is a potential concern regarding certification in software quality management?
What is a potential concern regarding certification in software quality management?
Signup and view all the answers
How do insufficient resources affect software quality?
How do insufficient resources affect software quality?
Signup and view all the answers
What is the primary advantage of the top-down integration approach?
What is the primary advantage of the top-down integration approach?
Signup and view all the answers
Which of the following is a disadvantage of the top-down integration approach?
Which of the following is a disadvantage of the top-down integration approach?
Signup and view all the answers
In bottom-up integration, what is baseline 1 comprised of?
In bottom-up integration, what is baseline 1 comprised of?
Signup and view all the answers
What role do stubs play in the top-down integration process?
What role do stubs play in the top-down integration process?
Signup and view all the answers
Which statement correctly describes a potential challenge when using the top-down approach?
Which statement correctly describes a potential challenge when using the top-down approach?
Signup and view all the answers
What is NOT a cause of software errors?
What is NOT a cause of software errors?
Signup and view all the answers
How many errors do software developers typically make per thousand lines of code?
How many errors do software developers typically make per thousand lines of code?
Signup and view all the answers
Which principle of testing states that testing cannot prove the absence of defects?
Which principle of testing states that testing cannot prove the absence of defects?
Signup and view all the answers
Which of the following is highlighted as a major economic impact of poor software quality in the USA?
Which of the following is highlighted as a major economic impact of poor software quality in the USA?
Signup and view all the answers
What is said about exhaustive testing?
What is said about exhaustive testing?
Signup and view all the answers
Which factor is NOT directly associated with the causes of software errors?
Which factor is NOT directly associated with the causes of software errors?
Signup and view all the answers
When should testing activities begin in the development life cycle?
When should testing activities begin in the development life cycle?
Signup and view all the answers
What is the approximate cost of software defects reported in the USA per year?
What is the approximate cost of software defects reported in the USA per year?
Signup and view all the answers
What does the left branch of the V-Model represent?
What does the left branch of the V-Model represent?
Signup and view all the answers
Which testing level in the V-Model ensures that groups of components interact as specified?
Which testing level in the V-Model ensures that groups of components interact as specified?
Signup and view all the answers
In the V-Model, when do test analysis and design begin?
In the V-Model, when do test analysis and design begin?
Signup and view all the answers
What is a primary focus of Quality Control (QC)?
What is a primary focus of Quality Control (QC)?
Signup and view all the answers
Which of the following models falls under iterative and incremental development models?
Which of the following models falls under iterative and incremental development models?
Signup and view all the answers
What does the acceptance test in the V-Model check for?
What does the acceptance test in the V-Model check for?
Signup and view all the answers
Which characteristic is NOT associated with the V-Model?
Which characteristic is NOT associated with the V-Model?
Signup and view all the answers
Which of the following statements about Quality Assurance (QA) is true?
Which of the following statements about Quality Assurance (QA) is true?
Signup and view all the answers
Which characteristic is NOT considered a non-functional characteristic?
Which characteristic is NOT considered a non-functional characteristic?
Signup and view all the answers
What is the primary goal of load testing?
What is the primary goal of load testing?
Signup and view all the answers
Which type of non-functional testing measures the response time of a system handling multiple requests?
Which type of non-functional testing measures the response time of a system handling multiple requests?
Signup and view all the answers
What defect is identified during the load testing example when concurrent requests exceed 490?
What defect is identified during the load testing example when concurrent requests exceed 490?
Signup and view all the answers
Which of the following is NOT a type of performance testing?
Which of the following is NOT a type of performance testing?
Signup and view all the answers
Why is speed an important factor in performance testing?
Why is speed an important factor in performance testing?
Signup and view all the answers
During a performance test, if a server does not respond in less than 2 seconds with 100 users, what does that indicate?
During a performance test, if a server does not respond in less than 2 seconds with 100 users, what does that indicate?
Signup and view all the answers
Which testing procedure aims to simulate requests to find the maximum performance capacity of a server?
Which testing procedure aims to simulate requests to find the maximum performance capacity of a server?
Signup and view all the answers
Study Notes
Software Quality Assurance and Testing - Course Introduction
- Lecturer: Dr. Nguyễn Quang Vũ
- Email: [email protected]
- Mobile: (+84) 901.982.982
- Course name: Software Quality Assurance Testing
- Course status for program: Required
- Knowledge block: Specialized knowledge
- Number of credits: 3 (60 sessions; 1 session = 50 minutes)
Student Tasks
- Attend more than 80% of sessions
- Students are responsible for doing and submitting all exercises, homework, assignments, and major assignments given by the instructor in class or at home on time.
- Use laptops in class only for learning purposes
- Regularly access the VKU's eLearning System (http://elearning.vku.udn.vn/) for up-to-date course information and submission of exercises, homework, assignments, and labs.
- Details of the course plan are in the Course Syllabus.
Chapter 1-2: SQA and SQA Introduction
- Quality assurance consists of procedures, techniques, and tools applied by professionals to ensure a product meets or exceeds pre-specified standards during its development cycle. (E. H. Bersoff)
- It is an essential activity for any business producing products used by others.
- It needs to be planned and systematic. (It does not happen just on its own)
- It needs to be built into the development process. (A natural outcome of software engineering)
- Continuous improvement is the overall goal.
- Four Ps in software development: People, Process, Project, and Product.
- The way to achieve software quality includes activities and measurements of process, and sets of artifacts of the product.
SQA - An Umbrella Activity
- SQA is an umbrella activity with methods and tools to maintain quality which includes testing, software configuration management, and formal technical reviews
Why SQA Activities Pay Off
- Cost of finding and fixing bugs in software decreases over time as they are discovered at earlier stages in development.
What is Software Quality?
- Simplistically, quality is an attribute of software that implies the software meets its specification.
- Software specifications are often incomplete or ambiguous
- Some quality attributes are difficult to specify
- Tension exists between some quality attributes, eg. efficiency vs. reliability
- Conformance to explicitly stated functional and performance requirements, explicitly documented development standards, and implicit characteristics that are expected of all professionally developed software
- Software requirements are the foundation from which quality is measured.
- Lack of conformance to requirements is lack of quality.
- Specified standards define a set of development criteria that guide the manner in which software is engineered.
- If the criteria are not met, lack of quality will almost surely result.
- If software conforms to its explicit requirements but fails to meet its implicit requirements, software quality is suspect.
Software Quality Assurance - SQA
- An organization must follow a three-prong approach to ensuring quality in the software product:
- Organization-wide policies, procedures, and standards must be established.
- Project-specific policies, procedures, and standards must be tailored from the organization-wide templates.
- Quality must be controlled to ensure appropriate procedures are followed for each project.
- Standards such as ISO 9000-3 and ANSI/IEEE standards exist to help organizations design an appropriate software quality assurance plan
- External entities can be contacted to verify the organization's standard compliance
SQA Activities
- Applying technical methods to achieve high-quality specifications and designs.
- Conducting formal technical reviews to uncover quality problems.
- Testing software with various test methods.
- Enforcing standards as part of controlling changes throughout the software's lifecycle.
- Maintaining records about SQA activities and reporting on quality.
- Keeping track and assessing the efficacy of methodological and procedural changes to improve software quality.
SQA Advantages
- Fewer latent defects lead to reduced effort and time spent during testing and maintenance.
- Higher reliability results in greater customer satisfaction.
- Maintenance costs can be reduced.
- Overall lifecycle cost of software is reduced.
SQA Disadvantages
- Implementation in smaller organizations might be difficult due to resources and requires cultural change.
- Requires specific expenditures not always explicitly budgeted for software engineering or QA
Quality Reviews
- The fundamental method for validating product or process quality throughout the software lifecycle.
- Identify needed improvements in a single person or team's product work and confirm parts where improvement is unnecessary or undesirable.
- Ensure work is as predictable and uniform as possible (makes management more manageable)
- Quality reviews can have multiple goals like: removing defects, assessing progress, and ensuring consistency and conformance
Cost Impact of Software Defects
- Errors from earlier stages are amplified as they move through later testing stages, incurring increasing costs to correct defects.
Defect Amplification and Removal
- Defects are often discovered in later testing phases at a proportionally higher cost than when the defect is identified earlier.
Software Configuration Management (SCM)
- Encompasses activities throughout the system life cycle to control changes in products and life-cycle artifacts to:
- control changes in plans, programs, specifications, documents/manuals, procedures, or data.
Configuration Item Identification and Description
- Each configuration item needs a unique name, classification, project identifier, and change/version information.
- Establish relationships among configuration items using diagrams like object diagrams or domain/use-case models
- A software library (database) stores, manages, and tracks configuration items.
Controlling Configuration Items: Baselines
- A baseline in software development is a point/phase (milestone) in which no more changes are allowed without formalizing the changes through a procedure as it moves into the next stage.
Controlling Configuration Items: Version Control
- Configuration items evolve through the software engineering process, creating versions.
- These changes can be illustrated by using an evolution graph.
- Version, branch, and variant are part of the configuration items' change history.
SCM Change Control
- Users/developers submit change requests.
- A change control authority evaluates requests.
- The authority decides whether to approve or reject the requests based on an engineering change order.
- Configuration items are checked-out, changed, then checked in after SQA and version controlled.
- Configuration items are available for use once approved and released
SCM Audit and Status Reporting
- An audit ensures that changes have been appropriately implemented.
- It is usually performed by a Quality Assurance (QA) group as a formal audit activity.
- Status reporting keeps all parties updated on the changes and their status; it works as a communication mechanism across the project.
- Status reporting can help determine what, when, and who made the changes during the project.
SCM Support
- The software library supports storage, labeling, identifying different versions, and tracking their statuses.
- Daily, weekly, and monthly reports are used for checking, maintaining, and managing the whole system.
SCM Benefits
- Reduces the effort to manage changes, leading to increased productivity
- Improves software integrity and security
- Enhances software quality (due to reduced error costs)
- Improves management control and testing of software
- Better records are kept and changes are tracked over time
Process Quality
- Unlike manufactured products, software development has factors such as creation; individual skills/experiences; and external factors (that contribute to quality).
- Software development processes are organization specific; people and technology may be more important than process.
- Insufficient resources always adversely affect quality
- ISO 9001/9000-3 focus on process management and customization for a particular organization:
- Certification is relatively easy and can be a marketing ploy.
Process Quality: The SEI CMM Model
- This model is used to improve organization's software development processes.
- Levels are defined to measure success and define the stages of improvement of process management required for each level.
People Quality: The PCMM Model
- This model intends to improve the knowledge and skills of people in an organization;
- Levels are defined to measure success and define the stages of development management required for each level.
Summary - Software Quality
- Quality software does not just happen.
- Quality assurance mechanisms should be built into the software development process.
- Developing quality software requires management support and involvement, gathering and using software metrics, and policies and procedures that everyone follows (even when things are tough).
- Testing is an important part of quality assurance, but it is not the only factor contributing to obtaining high-quality software.
Chapter 3: SQA Management
- What is Software Quality? Simplistically, quality is an attribute of software that implies the software meets its specification.
Product Quality: Design Metric
- For design components, maintainability is the key quality attribute.
- Maintainability is related to: cohesion, coupling, understandability, and adaptability.
Product Quality: Design Quality Metrics (structural fan-in/fan-out)
-Structural fan-in is the number of calls to a component by other components. -Structural fan-out is the number of components called by a component.
- A high fan-in indicates high coupling, and a high fan-out indicates high complexity in calling components.
- Informational fan-in/fan-out consider the number of parameters passed and access to shared data structures to compute a design structure quality index (DSQI) that can be used to measure stability.
- A software maturity index (SMI) is an index used to determine the degree of stability the software has during its lifecycle.
Product Quality: Design Quality Metrics (IEEE Standard 982.1-1988)
- Looks at subsystem and database properties (number of subsystems and degree of coupling, and number of attributes and classes.)
- Compute a design structure quality index (DSQI) to compare with past designs.
- Use a software maturity index (SMI) to assess stability; the higher the value the more stable the product.
Product Quality: Formal Approaches (Proving programs/specifications correct)
- Logically prove that requirements have correctly transformed into the program,
eg. Proving assertions about programs - Statistical Quality Assurance
- Categorize and determine causes of software defects. Identify causes of 80% of software defects, and concentrate on correcting them.
Project Quality: Reviews
- Requirements Capture
- Analysis
- Implementation
- Testing
- The process of reviewing requirements, analysis, and design; design walkthroughs of code, and test plans leading to early discovery of defects.
- Formal technical reviews will uncover 50-60% of all defects.
Software testing process
- Test Plans
- Test Evaluation and Reporting
- Test Control
Software Quality Assurance: Testing Related to Changes
- Confirmation Testing (Retesting): verifying a defect is fixed by running the test again.
- Regression Testing: testing all existing functionality after a code update to ensure no unforeseen issues arise in unaltered areas of the program.
Formal Technical Review (FTR)
- Structured quality assurance activity executed by practitioners. It's a review of any representation of the software.
- Ensures software design and code satisfy predefined standards.
The Review Meeting
- Components of the meeting include constraints, product, producer; review leader, reviewers, and recorder.
- Between 3 and 5 people are typically involved.
Review Reporting and Record Keeping
- Reports like Review Summary Report can help with identifying areas for correction within the product.
- A Review Issue List can be used as a checklist to guide corrections.
Guidelines for FTR
- Review the product, not the producer.
- Set an agenda and maintain it.
- Limit debate and rebuttal.
- Enunciate the problem areas, and don’t attempt to solve every problem that is noted.
- Take written notes.
- Limit the number of participants and insist upon advance preparation
- Develop a checklist for each product that is likely to be reviewed
- Conduct meaningful training for all reviewers
- Review your earlier reviews.
Reviewer's Preparation
- Be sure to understand the context of the material.
- Skim the product materials to understand the location and format of the information.
- Read the materials.
- Annotate a hard copy.
- Pose written comments as questions.
- Inform the review leader if one cannot prepare.
Software Reliability
- Probability of failure-free operation for a specified time in a specified environment.
- Informally, reliability is a measure of customer's perception based on the service the software provides; it's not an objective measure, but it needs an operational profile and consideration that serious consequences may vary for different errors.
Software Reliability Improvements
- Software reliability will improve by removing potential defects found in frequently used portions of the software.
- The removal of X% of faults does not mean there will be an X% improvement in reliability.
- Removing faults with the most serious consequences is the primary objective to improve reliability.
Chapter 4: Introduction to Software Testing
- What is software quality? - The degree to which a system, component, or process meets specified requirements.
- The cause of software errors:
- Time pressure
- Human error proneness
- Inexperienced or unskilled project participants
- Miscommunication
- Complexity of code, design, architecture, underlying problem, and technologies used
- New or unfamiliar technologies
- What is software testing? Testing's objective is to identify and reveal errors in the code, to prevent defects and to reduce risk
- Fundamental testing process stages: Begin; Planning and Analysis; Implementation and Design; Execution; Control; Evaluation; Exit Criteria; and Test Closure Activities
Test Planning
- Major tasks include identifying testing objectives, scope, approach, resources, and the like.
- Test control involves comparing the actual progress against plans, reporting status, and taking actions as needed
- Test control activities include measuring and analyzing results; monitoring development and progress of the project, and making decisions as needed
Analysis and design
- The purpose of analysis and design is to gain a better understanding of the testability of the basis of the test and the test objects.
- Document inputs and priorities to establish a base for test cases based on specifications.
- Develop test design specifications and use case models
- Define any needed test data to support test cases.
- Identify any needed infrastructure for the testing conditions
Implementation and Execution
- Develop test cases, and plan and choose priorities.
- Create test scripts for execution.
- Create any necessary test data.
- Verify and check the environment set up is correct.
Evaluating Exit Criteria and Reporting
- Evaluate exit criteria against defined objectives
- Check test logs for each test level against defined exit criteria
- Assess more tests or needed changes in exit criteria
- Prepare a test summary report for stakeholders
Test Closure Activities
- Collect and finalize test results/data.
- Archive test ware like scripts and test environments.
- Document lessons learned for future releases.
Fundamental Test Process
- Fix test design and repeat; fix components; or execute/fix test cases/scripts and repeat
- Plan, analyze, design, implement, and execute; control; evaluate exit criteria and reporting; and close the test activities
Chapter 5: Software Testing in the SDLC
- The Role of Software Testing in SDLC
- Definitions of Verification, Validation, QA, QC
- Test Levels: Component, Integration, System, Acceptance, and Regression Testing
- Types of Testing: Functional, Non-Functional, Structural, and Testing Related to Changes (Retesting and Regression Testing).
V-Model
- The two branches of the V show the relationships between development and testing activities.
- The left branch represents the development process in which requirements are captured, functional and technical specifications are designed, and detailed component specifications are generated; these specifications guide code implementation.
- The right branch represents different levels of testing including Verification and Validation (V&V), implementation, and integration.
Verification & Validation
- Verification (the design): Are we building the product right? To ensure that work products meet their specified requirements.
- Validation (the product): Are we building the right product? To ensure that the product actually meets the user's needs and that specifications were correct in the first place.
Quality Assurance vs. Quality Control
- Quality Control is about fulfilling the quality requested. Aims to identify and correct defects, focusing on full software testing life cycle (testing, and measuring). QC activities are only part of the range of QA activities.
- Quality Assurance provides assurance the quality request will be achieved, preventing causes of defects early in the development process
Test Levels
- Test level (test stage) is a group of activities that are organized and managed together in a project. Test levels include: component testing; integration testing; system testing; acceptance testing and regression testing
Component Testing
- Unit testing, module testing, program testing, or testing individual program units (procedures, functions, methods, or classes.)
- The goal of component (unit) testing is ensuring the code written for the unit meets its specification before integrating into other units.
Integration Testing
- Integration testing determines whether software functions correctly as a whole between components and systems.
- Various integration strategies exist (e.g. big-bang, incremental, top-down, bottom-up, minimum capability/functional, and thread integration).
Big-Bang Integration
- A strategy that tests all components only when they are integrated together.
- This strategy is time consuming during development and fixing errors.
Incremental Integration
- The components/modules are integrated and tested as they are completed to provide a working system sooner.
Top-Down Integration
- A strategy to start with the primary/high-level components/modules and integrate with lower-level components/modules when they are ready.
- Testing begins with the top most components and progresses to lower levels; stubs are used to simulate missing components.
Bottom-Up Integration
- A strategy to test lower-level components/modules first before integrating them into higher-level components/modules in a series. The modules are combined and tested.
Minimum Capability Integration
- A baseline is created as the lowest level component is finished. Then, components are combined to a baseline. At each stage, component testing progresses until all components are integrated and tested in a system configuration.
Thread Integration
- A strategy that follows the order of events to determine the testing order.
Integration Guidelines
- Minimize the amount of support software needed
- Integrate each component once
- Each baseline should produce an easily verifiable result.
- Integrate small numbers of components at a time.
- Combine simple related components first.
Integration Planning
- Plan integration in the architectural phase.
- The integration order determines the build order.
- Enables parallel component development and integration testing to save time.
Testing-Related Changes
- Confirmation testing (retesting): testing a defect again after it has been fixed.
- Regression testing: testing existing functionality in the system when the code or environment changes.
System Testing
- System testing aims to ensure the integrated system fulfills its requirements or specifications through various techniques.
- Tests may include those related to risks, and requirements, and business process; use cases; or system interaction with the operating systems, or system resources.
Acceptance Testing
- Acceptance testing evaluates the software product's performance according to user requirements and the contract or legal or regulatory standards, and determines if it can be released to customers.
- Different types of acceptance testing include contract and regulation acceptance testing; user acceptance testing (UAT); operational acceptance testing, and alpha and beta testing.
Alpha and Beta Testing
- Alpha testing simulates potential users and is carried out by an internal team at the development organization.
- Beta testing is a form of external acceptance testing and carried out with a group of potential users at their own locations.
- Alpha testing is conducted prior to beta testing.
Test Types
- Functional testing: is focused on functionality; testing of functions.
- Non-functional testing determines how well a system works, not whether it meets a spec.
- Structural testing determines the structure of the software system, component, or module
- Testing related to changes (Testing related to modifications to software) include retesting which is confirming that a defect is really fixed and regression testing which covers the whole software to ensure no other issues have arisen.
Functional Testing
- Functional testing checks functions to match specifications/requirements; It is also called specification testing
Non-Functional Testing
- Testing non-functional characteristics such as reliability, usability, efficiency, maintainability, and portability
Performance Testing
- The objective of performance testing is to measure how efficiently all components and the whole system functions.
- The methods of performance testing include Load testing, Stress testing, Endurance testing, and Spike testing
Load testing
- Is testing performed on a system/component under the expected load.
Stress Testing
- Testing the system or component at or beyond the limits of its anticipated work loads.
Endurance Testing/Soak Testing
- Tests the performance under specific load conditions over an extended or longer amount of time.
Spike Testing
- Testing of the system or component under a sudden load or a very high load in a short duration of time
Document Testing
- Focused on the technical accuracy and readability of user manuals, tutorials, and online help. Tests like spelling or grammar checks along with manual reviews are included to eliminate any sort of inconsistencies in documents.
Installation Testing
- Ensures that the software is correctly installed along with all its associated features as per expectations.
Reliability Testing
- Determines if the software fulfills its intended purpose in a specified environment and operates without errors and failures for a given time, this involves load testing and memory management testing
Security testing
- Checks that data and functionality are protected in the system and maintained by following all safety-related aspects and requirements
Structural testing
- Checks the structure of the software system or component such as in source code, program code, or modules, and looks into the parts/components of the software and detects error from the source. Structural testing also covers code coverage aspects. This approach may use tools to check the code coverage percentage of executable elements such as statements, or decisions, which have been covered by several test suites
Testing related to changes
- Confirmation testing(retesting) confirms that a discovered defect has been repaired by running test cases again.
- Regression testing ensures existing code/environment changes do not introduce or uncover new issues/defects in the unchanged areas of the software
Chapter 6: Black-Box Testing
- What is black box testing? A method for determining whether the software's components or whole system fulfill the specification provided based on testing of functions and the user's experience, not the internal workings.
- Black-box testing techniques: Equivalence Class Testing; Boundary Value Testing; Combinatorial Testing; State Transition Testing; Decision Table Testing, and Use Case Testing
Equivalence Class testing
- Divide the input domain of a program into classes of input data (valid, invalid)
- Determine test cases based on these partitions
Boundary value testing
- Test cases at the boundaries (e.g. minimum, maximum, just above minimum, just below maximum)
- Errors often occur near extreme values.
Combinatorial testing/Pairwise Testing/All-Pairs Testing
- A black-box technique used to design test cases for all the possible combinations of input parameters.
- These tests are effective for testing discrete combinations yet still smaller than exhaustive tests.
State Transition Testing
- A testing method used for testing/tracking various states of a scenario/system and identifying the possible transitions between them. This testing technique will be helpful for testing transitions in systems.
Studying That Suits You
Use AI to generate personalized quizzes and flashcards to suit your learning preferences.
Related Documents
Description
Explore the essential concepts of Software Configuration Management (SCM) through this quiz. Gain insights into its benefits, functions, and the impact on software quality and development processes. Test your knowledge on common practices in SCM.
