Software Quality Assurance CSE3423 PDF

SOFTWARE QUALITY ASSURANCE CSE3423 SOFTWARE AND OUR EVERDAY LIFE CSE3423 TABLE OF CONTENTS Lecturers CHAPTERS Course Content and Schedule Course Objectives Course Learning Outcome Assesment References Let's Connect With Us! Lecturers PROFESOR...

SOFTWARE QUALITY ASSURANCE CSE3423 SOFTWARE AND OUR EVERDAY LIFE CSE3423 TABLE OF CONTENTS Lecturers CHAPTERS Course Content and Schedule Course Objectives Course Learning Outcome Assesment References Let's Connect With Us! Lecturers PROFESOR DATO` Ts. Dr. Ily Amalina REESE MILLER TS. DR. AZIZ BIN Binti Ahmad Sabri DERAMAN K2 K1 COURSE CONTENT AND SCHEDULE CHAPTER 1 CHAPTER 2 CHAPTER 3 Introduction ( 4 + 2 Quality Assurance Quality Check (4 + 2 + + 6 hrs) Frameworks (4 + 2 + 6 hrs) 6 hrs) COURSE CONTENT AND SCHEDULE CHAPTER 4 CHAPTER 5 Software Quality Assurance Certification (4 + 2 + Managment (4 + 2 + 6 6 hrs) hrs) Course Objectives 1 2 3 OBJECTIVE OBJECTIVE OBJECTIVE Expose students to Provide an Involve students in describe factors that understanding of applying the methods in influence software techniques for the the software quality quality. measurements of control, assurance, software quality. inspection and certification  Course Learning Outcomes (CLO) This course introduces function, procedure, and method that are exercised in controlling and assuring software quality. These include elements and the CLO2 role of quality assurance, quality inspection, software testing and also Identify appropriate software quality methods for use in the development introduces some basic tools in software of a software or system. (C4, PLO3) quality. CLO1 CLO3 Describe different techniques in Demonstrate strong teamwork and determining the quality of software commitment in completing the (C2, PLO1) assigned project. (A3, PLO5) Assesment 20 15 10 Presentation Presentation Reportt Reportt 5 0 CHAPTER CHAPTER 1 CHAPTER 2 Topic 1: Introduction Topic 2: Quality Assurance Framework and Standard Definition of Quality and Quality of Attributes Quality Assurance Component Software Quality Assurance Definition ISO / IEC Quality Standards Quality & Economy Capability Maturity Model Integration Quality Factors Software Configuration Management Basic Quality Tools Software Quality Assurance Plan CSF3423 CHAPTER CHAPTER 3 CHAPTER 4 Topic 3: Quality Check Topic 4: Quality Assurance Management Checking Type Quality Organization and Management Inspection Objectives QA Personnel :Roles and Principles of Examination Responsibilities Inspection Behavior Project Progress Control Inspection Exercises Software Quality Metrics Inspection Report Software Quality Costs CSF3423 CHAPTER CHAPTER 5 Topic 5: Software Certification Modelling Software Product Quality Software Process Quality Certification Model (eg. MSTB & Korean Testing Laboratory) Software Certification in Practice CSF3423 References THANK YOU An introduction to Quality Topic 1: CSE3423 SOFTWARE QUALITY ASSURANCE Ily Amalina Ahmad Sabri Aziz Deraman Standards in Action www.bsieducation.org/standardsinaction 15 An introduction to Quality Introduction to Quality Expanding the quality myth Author: Dr Rhys Rowland-Jones Standards in Action www.bsieducation.org/standardsinaction 16 An introduction to Quality Session Plan Different views of quality General definitions of quality Some issues facing the quality profession Views of quality Costs of quality Dimensions of quality Standards in Action www.bsieducation.org/standardsinaction 17 An introduction to Quality The first question to ask– What is Quality? How would you describe what “Quality” means? Standards in Action www.bsieducation.org/standardsinaction 18 An introduction to Quality QUALITY Degree to which a set of inherent characteristics fulfils requirements ISO 9000:2000 Standards in Action www.bsieducation.org/standardsinaction 19 An introduction to Quality Phases of Quality Assurance Inspection and Inspection corrective Quality built before/after action during into the production production process Acceptance Process Continuous sampling control improvement The least The most progressive progressive Standards in Action www.bsieducation.org/standardsinaction 20 An introduction to Quality QUALITY DOES NOT OCCUR BY ACCIDENT What does the customer actually want? – Identify, understand and agree customer requirements How are you going to meet those requirements? – Plan to achieve them Standards in Action www.bsieducation.org/standardsinaction 21 An introduction to Quality The Demming Cycle W.Edwards Demming Plan Control Act & Do Improvement Check Standards in Action www.bsieducation.org/standardsinaction 22 An introduction to Quality Some issues facing the quality profession How to define quality from the customer’s perspective? Keeping up with the constant increases in the level of quality of today’s goods and services. The particular difficulties encountered in managing service quality. How does the organization identify the quality dimensions that are most important to its customers? Standards in Action www.bsieducation.org/standardsinaction 23 An introduction to Quality Some issues facing the quality profession Being able to avoid the costs of poor quality products and services. Being able to deal with the shift in balance of power to consumers from producers through globalization. Recognizing that customer loyalty is increasingly based on quality. Getting ‘leaner’ by achieving higher levels of productivity. Standards in Action www.bsieducation.org/standardsinaction 24 An introduction to Quality Expressing Dissatisfaction Public action can be Seeking redress directly from Takes the firm action Taking legal action A dissatisfied A complaint to business, private, customer or governmental agencies Private action Stop buying the product or boycott the seller Takes Warn friends about the product no action and/or seller Standards in Action www.bsieducation.org/standardsinaction 25 An introduction to Quality Customer Feedback and Word-of-Mouth The average business only hears from 4% of its customers who are dissatisfied with its products or services. Of the 96% who do not bother to complain, 25% of them have serious problems. The 4% complainers are more likely to stay with the supplier than are the 96% non- complainers. About 60% of the complainers would stay as customers if their problem was resolved and 95% would stay if the problem was resolved quickly. A dissatisfied customer will tell between 10 and 20 other people about their problem. A customer who has had a problem resolved by a company will tell about 5 people about the situation. Standards in Action www.bsieducation.org/standardsinaction 26 An introduction to Quality An Approach to Viewing Quality. Slack et al 2004 The transcendent approach views quality as synonymous with innate excellence e.g. Rolls Royce, Rolex, The Hilton. The manufacturing-based approach assumes quality is all about making or providing error-free products or services e.g. Audi’s ‘vorsprung durch technik’. The user-based approach assumes quality is all about providing products or services that are fit for their purpose e.g. it does what it says on the tin! The product-based approach views quality as a precise and measurable set of characteristics e.g. 0-60 in 4.3 seconds. The value-based approach defines quality in terms of value’ e.g. supermarket ‘value’ ranges. Standards in Action www.bsieducation.org/standardsinaction 27 An introduction to Quality Quality Characteristics of Goods and Services Functionality - how well the product or service does the job for which it was intended. Appearance - aesthetic appeal, look, feel, sound and smell of the product or service. Reliability - consistency of product or service’s performance over time. Durability - the total useful life of the product or service. Recovery - the ease with which problems with the product or service can be rectified or resolved. Contact - the nature of the person-to-person contacts that take place. Standards in Action www.bsieducation.org/standardsinaction 28 An introduction to Quality Internal and External Benefits of Quality Internal Benefits External Benefits Customer gets correct Reduces costs product or service Increases dependability Increases speed Correct specifications Boosts moral Appropriate intangibles Increases customer retention Increases profit Customer satisfaction Customer retention Standards in Action www.bsieducation.org/standardsinaction 29 An introduction to Quality The ‘Iceberg’ theory – how much is immediately visible? Scrap, waste Loss of customers Standards in Action www.bsieducation.org/standardsinaction 30 An introduction to Quality British Standards on Quality Costs BS 6143 Part 1 BS 6143 Part 2 Prevention Appraisal Failure Model (PAF) Process Cost Model (PCM) Standards in Action www.bsieducation.org/standardsinaction 31 An introduction to Quality Costs of Quality Failure “Defects are not free, someone makes them and gets paid for the privilege” COST OF INTERNAL FAILURE – Scrapped materials, goods and services – Rework/ retest – Reduced capacity/ yield/ increased downtime – Rescheduling – Service delays – Disruption to the service process. – Focus is on troubleshooting not improvement COST OF EXTERNAL FAILURE – Warranty and servicing costs – Product liability / Litigation – Complaints and their administration – Loss of customer goodwill – Inconvenience to other customers Standards in Action www.bsieducation.org/standardsinaction 32 An introduction to Quality The Economic Costs of Quality COST OF PREVENTION Quality planning Design of quality system Staff quality training and development Preventative maintenance Supplier development training Administering quality procedures (e.g. ISO 9001) Time spent problem - solving, improving process Measurement of customer satisfaction during process COST OF APPRAISAL Testing and Inspection of supplier goods and services Testing and Inspection of internal service processes Measurement of customer satisfaction after process Quality Audits Standards in Action www.bsieducation.org/standardsinaction 33 An introduction to Quality The Ferdows and DeMayer Sandcone Model of Operational Improvement: Cost Flexibility Speed Dependability Quality Quality Quality + Dependability Quality + Dependability + Speed Quality + Dependability + Speed + Flexibility Quality + Dependability + Speed + Flexibility + Cost (FERDOWS & DeMAYER Adapted from Slack et al 2004) Standards in Action www.bsieducation.org/standardsinaction 34 An introduction to Quality A QUICK THINKING: Quality Characteristics Consider how the quality characteristics (functionality, reliability, appearance, durability, recovery and contact) relate to your organisation’s (UMT) main products / services? (one example) Note your answers – now ask your friends the same question and compare your answers. Are they similar? – you can share your finding in the next lecture Standards in Action www.bsieducation.org/standardsinaction 35 An introduction to Quality The Dimensions of Quality. The meaning of Quality Producer’s perspective Consumer’s perspective Quality of conformance Quality of design Production Conformance to Quality Marketing specifications characteristics Cost Price Standards in Action Fitness for www.bsieducation.org/standardsinaction consumer use 36 An introduction to Quality QUALITY MANAGEMENT SYSTEM Management system to direct and control an organisation with regard to quality ISO 9000:2000 Standards in Action www.bsieducation.org/standardsinaction 37 An introduction to Quality PURPOSE OF ISO 9001:2000 “ISO 9001 specifies the requirements for a quality management system that may be used for internal application by organizations, certification, or contractual purposes.” Standards in Action www.bsieducation.org/standardsinaction 38 An introduction to Quality Summary Quality has several dimensions Quality is not only a system There are costs to poor quality Quality is a continuous journey Standards in Action www.bsieducation.org/standardsinaction 39 Topic 1: CSE3423 SOFTWARE QUALITY ASSURANCE Ily Amalina Ahmad Sabri Aziz Deraman Introduction What is Software Quality? 41 CSE3423 Introduction (cont.) 42 CSE3423  Quality is hard to Quality is fitness define or describe in for purpose or abstract terms, but can meeting’s user needs User view be recognized if it is present Transcendental view The focus is on General Quality Views inherent characteristics Kitchenham and Pleeger, 1996; Pfleeger et al., 2002 in the product itself Quality is the customer’s Quality means Product view willingness to conformance pay for a to process software Value-based view standards Manufacturing view 43 CSE3423  What is software?  (Istilah Dewan Bahasa & Pustaka (DBP))  program atau atur cara komputer yg dapat digunakan dgn sistem komputer tertentu: doktor lebih mudah menentukan rawatan dan pencegahan penyakit jantung dgn menggunakan ~ komputer yg direka khas; ~ bersepadu perisian yg menggabungkan pelbagai ciri yg biasanya wujud secara berasingan dlm satu perisian yg besar dan bersepadu; ~ kongsi perisian atau dokumen yg diedarkan oleh pembuat perisian untuk dikongsi bersama oleh pengguna komputer; 44 CSE3423  What is software? a general term for the various kinds of programs used to operate computers and related devices.  Software can be thought of as the variable part of a computer and hardware the invariable part  often divided into application software (programs that do work users are directly interested in) and  system software (which includes operating systems and any program that supports application software - middleware used to describe programming that mediates between application and system software) 45 CSE3423 What is software? Software - IEEE definition: Computer programs, procedures, and possibly associated documentation and data pertaining to the operation of a computer system. Procedures Documentation Data necessary Computer for operating Programs the software system Software 46 CSE3423 People’s quality expectations The software systems Do the right must do what they are things supposed to do. They must perform Do the these specific tasks things right correctly and satisfactorily. 47 CSE3423 Quality problems Size Flexibility / adaptability Major Complexity expected difficulties Environmental stress / constraints 48 CSE3423 Quality : Views and attribute View Attribute Correctness Other Customer (external) Failures: Maintainability Reliability Readability Safety, etc. Portability Performance Installability Usability, etc. Developer (internal) Faults: Design Count Size Distr Change Class, etc. Complexity Presentation Control Data, etc. ISO (2001)classifies the quality attributes in four classes: process quality, internal and external quality (Technical Parameter), and quality in use (User Parameter) 49 CSE3423 50 CSE3423 51 CSE3423 Quality characteristics (ISO 9126) Functionality Suitability Accuracy Interoperability Security Portability Reliability Adaptability Maturity Installability Fault tolerance Conformance Security Replaceability Characteristics of Software Quality Maintainability Efficiency Analyzability Time behavior Changeability Resource Stability behavior Testability Usability Understandability Learnability Operability 52 CSE3423 Quality characteristics (cont.) A set of attributes that bear on the existence of a set of functions and their Functionality specified properties. The functions are those that satisfied stated or implied needs. Reliability A set of attributes that bear on the capability of software to maintain its level of performance under stated conditions for a stated period of time. Usability A set of attributes that bear on the effort needed for use, and on the individual assessment of such use, by a stated or implied set of users. A set of attributes that bear on the relationship between the level of Efficiency performance of the software and the amount of resources used, under stated conditions. Maintainability A set of attributes that bear on the effort needed to make specified modifications. Portability A set of attributes that bear on the ability of software to be transferred from one environment to another. 53 CSE3423  Software Quality Technical Quality Parameters  Correctness - lack of bugs and defects  measured in terms of defect rate (# bugs per line of code)  Reliability - does not fail or crash often  measured in terms of failure rate (#failures per hour)  Capability - does all that is required  measured in terms of requirements coverage (% of required operations implemented)  Maintainability - is easy to change and adapt to new requirements  measured in terms of change logs (time and effort required to add a new feature) and impact analysis (#lines affected by a new feature)  Performance - is fast and small enough  measured in terms of speed and space usage (seconds of CPU time, Mb of memory, etc.)  © J.S. Bradbury, J.R. Cordy 54 CSE3423  Software Quality User Quality Parameters  Usability - is sufficiently convenient for the intended users  measured in terms of user satisfaction (% of users happy with interface and ease of use)  Installability - is convenient and fast to install  measured in terms of user satisfaction (#install problems reported per installation)  Documentation - is well documented  measured in terms of user satisfaction (% of users happy with documentation)  Availability - is easy to access and available when needed  measured in terms of user satisfaction (% of users reporting access problems)  © J.S. Bradbury, J.R. Cordy 55 CSE3423  Classification of Software Quality Factors McCall’s Factor Model  Classifies all software requirements into 11 factors.  The factors are grouped in 3 categories: 1. Product operation factors (Factors that deal with requirements that directly affect the daily operation of the software). 2. Product revision factors (Factors that deal with requirements that affect software maintenance). 3. Product transition factors (Factors that deal with requirements that affect the adaptation of software to other platforms, environments, interaction with other software). © J.S. Bradbury, J.R. Cordy 56 CSE3423  McCall’s Factor Model Tree © J.S. Bradbury, J.R. Cordy 57 CSE3423 Software error, fault and failure An incorrect step, process, or data definition in a computer program. Error Fault A human action that produces an incorrect result Failure The inability of a system or component to perform its required functions within specified performance requirements. Defect Sofware Failure 58 CSE3423 Causes of software errors Faulty requirements definition Client- Documentation developer errors communication failures Deliberate Procedure deviations from errors software requirements Software Errors Shortcomings Logical design of the testing errors process Non- compliance with documentation Coding errors and coding instructions 59 CSE3423 Causes of software errors 1. Faulty definition of requirements  Erroneous requirement definitions  Absence of important requirements  Incomplete requirements  Unnecessary requirements included  2. Client-developer communication failures  Misunderstanding of client requirements presented in writing,orally, etc.  Misunderstanding of client responses to design problems 60 CSE3423 Causes of software errors 3. Deliberate deviations from software requirements  Reuse of existing software components from previous projects without complete analysis  Functionality omitted due to budget or time constraints  Improvements” to software that are not in requirements 4. Logical design errors  Errors in interpreting the requirements into a design (e.g. errors  in definitions of boundary conditions, algorithms, reactions to illegal operations,...) 5. Coding errors  Errors in interpreting the design document, errors related to incorrect use of programming language constructs, etc. 61 CSE3423 Causes of software errors 6. Non-compliance with documentation and coding instructions  Errors resulting from other team members coordinating with non- complying member’s code  Errors resulting from individuals trying to understand/maintain/ test non- complying member’s code 7. Shortcomings of the testing process  Incomplete test plan  Failure to report all errors/faults resulting from testing  Incorrect reporting of errors/faults  Incomplete correction of detected errors 62 CSE3423 Causes of software errors 8. Procedural errors  Incorrect procedures related to user activities that occur in the software 9. Documentation errors  Errors in the design documents or code comments  Errors in user manuals for software 63 CSE3423 Defect related concepts and relations e1 f1 e2 Usage Scenarios and Results (input/output) Input to S/w Dev. Software System x: failure e: error source f: fault e3 x1 f2 e4 f3 x2 e5 f4 e6 Legend: “a” causes “b”: a b 64 CSE3423 Correctness-centered properties View Attribute Correctness Other Customer / External Failure-related properties Maintainability (user/customer) Readability Portability Performance Installability Usability, etc. Developer / Internal Fault-related properties Design (developer, manager, tester, etc.) Size Change Complexity Presentation Control Data, etc. 65 CSE3423 Failure-related properties and measurement Failure properties Include information about the specific failures, what they are, how they and direct failure occur, etc. Can be measured directly by examining failure count, distribution, measurement density, etc. Failure likelihood How often and how likely a failure is going to occur is of critical concern to software users and customers. and reliability This likelihood is captured in various reliability measures, where reliability can be defined as the probability of failure-fee operations for a measurement specific time period or for a given set of input (Musa et al., 1987, Lyu, 1995, Tian 1998). Failure severity The failure impact is also a critical concern for users and customers of many software products and services, especially if the damage caused by measurement and failures could be substantial. Accidents, which are defined to be failures with severe consequences, safety assurance need to be avoided, contained, or dealt with to ensure the safety for the personnel involved and to minimize other damages. 66 CSE3423 Quality in Software Engineering Functional stage Focus on providing the automated functions to replace what had been done manually before. Schedule stage Focus on introducing important features and new systems on a timely and orderly basis to satisfy urgent user needs. Cost stage Focus on reducing the price to stay competitive accompanied by the widespread use of personal computers. Reliability stage Focus on managing user’s quality expectations under the increased dependency on software and high cost or severe damages associated with software failures. 67 CSE3423  What is Software Quality? Software quality may include many different attributes and may be defined and perceived differently based on people’s different roles and responsibilities. According to Pressman’s definition, is the degree of conformance to specific functional requirements, specified software quality standards, and Good Software Engineering Practices (GSEP). The degree to which a system, component, or process meets specified requirements [based on Philip B. Crosby’s definition, 1979]“ IEEE Definition The degree to which a system, component or process meets customer or user needs or expectations. [based on Joseph M. Juran, 1988] 68 CSE3423  Quality Assurance – What is it? Achieving Quality  Product and software quality does not happen by accident, and is not something that can be added on after the fact  To achieve quality, we must plan for it from the beginning,and continuously monitor it day to day  This requires discipline  Methods and disciplines for achieving quality results are the study of Quality Assurance or QA 69 CSE3423 © J.S. Bradbury, J.R. Cordy  Quality Assurance – What is it? Three General Principles of QA  Know what you are doing  Know what you should be doing  Know how to measure the difference 70 CSE3423 Software Quality Assurance QA Principle 1: Know What You Are Doing  In the context of software quality, this means continuously understanding what it is you are building, how you are building it and what it currently does  This requires organization, including having a management structure, reporting policies, regular meetings and reviews, frequent test runs, and so on  We normally address this by following a software process with regular milestones, planning, scheduling, reporting and tracking procedures 71 CSE3423 Software Quality Assurance QA Principle 2: Know What You Should be Doing  In the context of software quality, this means having explicit requirements and specifications  These must be continuously updated and tracked as part of the software development and evolution cycle  We normally address this by requirements and use-case analysis, explicit acceptance tests with expected results, explicit , frequent user feedback  Particular procedures and methods for this are usually part of our software process 72 CSE3423 Software Quality Assurance QA Principle 3: Know How to Measure the Difference  In the context of software quality, this means having explicit measures comparing what we are doing to what we should be doing  Achieved using four complementary methods:  Formal Methods - consists of using mathematical models or methods to verify mathematically specified properties  Testing - consists of creating explicit inputs or environments to exercise the software, and measuring its success  Inspection- consists of regular human reviews of requirements, design, architecture, schedules and code  Metrics- consists of instrumenting code or execution to measure a known set of simple properties related to quality 73 CSE3423 Formal Methods  Formal methods include formal verification (proofs of correctness), abstract interpretation (simulated execution in a different semantic domain, e.g., data kind rather than value), state modelling (simulated execution using a mathematical model to keep track of state transitions), and other mathematical methods  Traditionally, use of formal methods requires mathematically sophisticated programmers, and is necessarily a slow and careful process, and very expensive  In the past, formal methods have been used directly in software quality assurance in a small (but important) fraction of systems  Primarily safety critical systems such as onboard flight control systems, nuclear reactor control systems, embedded systems such as automobile braking systems andmedical equipment, and so on…however, today formal methods is becoming more widely used (as we will see later). 74 CSE3423  Testing Focus of the Course  The vast majority (over 99%) of software quality assurance uses testing, inspection and metrics instead of formal methods  Example: at the Bank of Nova Scotia, over 80% of the total software development effort is involved in testing! Testing  Testing includes a wide range of methods based on the idea of running the software through a set of example inputs or situations and validating the results  Includes methods based on requirements (acceptance testing), specification and design (functionality and interface testing), history (regression testing), code structure (path testing), and many more 75 CSE3423  Inspection and Metrics Inspection Inspection includes methods based on a human review of the software artifacts Includes methods based on requirements reviews, design reviews, scheduling and planning reviews, code walkthroughs, and so on Helps discover potential problems before they arise in practice 76 CSE3423  Metrics Metrics  Software metrics includes methods based on using tools to count the use of features or structures in the code or other software artifacts, and compare them to standards  Includes methods based on code size (number of source lines), code complexity (number of parameters, decisions, function points, modules or methods), structural complexity (number or depth of calls or transactions), design complexity, and so on  Helps expose anomalous or undesirable properties that may reduce reliability and maintainability 77 CSE3423 Achieving Software Quality Software Process Software quality is achieved by applying these techniques in the framework of a software process There are many software processes proposed, of which eXtreme Programming is one of the more recent 78 CSE3423 Achieving Software Quality Standards and Disciplines  Because of the importance of quality in software production and the relatively bad track record of the past, many standards and disciplines have been developed to enforce quality practice  Examples are Total Quality Management (TQM), the Capability Maturity Model (CMM), the Personal Software Process (PSP), the Microsoft Shared Development Process (SDP), the Rational Unified Process (RUP), and ISO 9000 79 CSE3423 Achieving Software Quality Standards and Disciplines  The ISO/IEC 9126 standard describes a software quality model which categorizes software quality into six characteristics (factors) which are sub- divided into sub-characteristics (criteria).  The characteristics are manifested externally when the software is used as a consequence of internal software attributes (Examples of external metrics are given in ISO 9126-2.)  The internal software attributes are measured by means of internal metrics (e.g., monitoring of software development before delivery). Examples of internal metrics are given in ISO 9126-3. 80 CSE3423  Achieving Software Quality A new standard for software product quality ISO 25010 standard for software product quality was published by the International Organization for Standardization (ISO) as a successor of the ISO 9126 standard that has been in use since 2001 The quality model of the Software Improvement Group, has been updated to reflect the change from ISO 9126 to ISO 25010. 81 CSE3423  Summary  We’ve seen what software means  We’ve seen what quality means, how it applies to software, and what methods we can use to achieve it References  Kan, Metrics and Models in Software Quality Engineering, ch.1  Galin, Software Quality Assurance: From Theory Implementation, ch 2 & 3  The Software Quality Page (see link in WebCT) © J.S. Bradbury, J.R. Cordy (lecture notes) 82 CSE3423 Topic 1: CSE3423 SOFTWARE QUALITY ASSURANCE Ily Amalina Ahmad Sabri Aziz Deraman  Seven Quality Tools The Seven Tools – Histograms, Pareto Charts, Cause and Effect Diagrams, Run Charts, Scatter Diagrams, Flow Charts, Control Charts 12/31/2024 CSE3423 84  Ishikawa’s Basic Tools of Quality Kaoru Ishikawa developed seven basic visual tools of quality so that the average person could analyze and interpret data. These tools have been used worldwide by companies, managers of all levels and employees. 12/31/2024 CSE3423 85  Histograms Slide 1 of 3 Histogram Defined – A histogram is a bar graph that shows frequency data. – Histograms provide the easiest way to evaluate the distribution of data. 12/31/2024 CSE3423 86  Histograms Slide 2 of 3 Creating a Histogram – Collect data and sort it into categories. – Then label the data as the independent set or the dependent set. The characteristic you grouped the data by would be the independent variable. The frequency of that set would be the dependent variable. – Each mark on either axis should be in equal increments. – For each category, find the related frequency and make the horizontal marks to show that frequency. 12/31/2024 CSE3423 87  Histograms Slide 3 of 3 Examples of How Histograms Can Be Used – Histograms can be used to determine distribution of sales. – Say for instance a company wanted to measure the revenues of other companies and wanted to compare numbers. 12/31/2024 CSE3423 88  Pareto Charts Slide 1 of 4 Pareto Chart Defined – Pareto charts are used to identify and prioritize problems to be solved. – They are actually histograms aided by the 80/20 rule adapted by Joseph Juran. Remember the 80/20 rule states that approximately 80% of the problems are created by approximately 20% of the causes. 12/31/2024 CSE3423 89  Pareto Charts Slide 2 of 4 Constructing a Pareto Chart – First, information must be selected based on types or classifications of defects that occur as a result of a process. – The data must be collected and classified into categories. – Then a histogram or frequency chart is constructed showing the number of occurrences. 12/31/2024 CSE3423 90  Pareto Charts Slide 3 of 4 An Example of How a Pareto Chart Can Be Used – Pareto Charts are used when products are suffering from different defects but the defects are occurring at a different frequency, or only a few account for most of the defects present, or different defects incur different costs. What we see from that is a product line may experience a range of defects. The manufacturer could concentrate on reducing the defects which make up a bigger percentage of all the defects or focus on eliminating the defect that causes monetary loss. Actual chart is on the next slide » Example and chart were obtained from: 12/31/2024 CSE3423 91  Pareto Charts Slide 4 of 4 12/31/2024 CSE3423 92  Cause and Effect Diagrams Slide 1 of 4 Cause and Effect Diagram Defined – The cause and effect diagram is also called the Ishikawa diagram or the fishbone diagram. – It is a tool for discovering all the possible causes for a particular effect. – The major purpose of this diagram is to act as a first step in problem solving by creating a list of possible causes. 12/31/2024 CSE3423 93  Cause and Effect Diagrams Slide 2 of 4 Constructing a Cause and Effect Diagram – First, clearly identify and define the problem or effect for which the causes must be identified. Place the problem or effect at the right or the head of the diagram. – Identify all the broad areas of the problem. – Write in all the detailed possible causes in each of the broad areas. – Each cause identified should be looked upon for further more specific causes. – View the diagram and evaluate the main causes. – Set goals and take action on the main causes. 12/31/2024 CSE3423 94  Cause and Effect Diagrams Slide 3 of 4 An Example of When a Cause and Effect Diagram Can Be Used – This diagram can be used to detect the problem of incorrect deliveries. Diagram on next slide – Diagram obtained from: – When a production team is about to launch a new product, the factors that will affect the final product must be recognized. The fishbone diagram can depict problems before they have a chance to begin. 12/31/2024 CSE3423 95  Cause and Effect Diagrams Slide 4 of 4 Diagram of the Incorrect Deliveries Example: 12/31/2024 CSE3423 96  Scatter Diagrams Slide 1 of 4 Scatter Diagrams Defined – Scatter Diagrams are used to study and identify the possible relationship between the changes observed in two different sets of variables. 12/31/2024 CSE3423 97  Scatter Diagrams Slide 2 of 4 Constructing a Scatter Diagram – First, collect two pieces of data and create a summary table of the data. – Draw a diagram labeling the horizontal and vertical axes. It is common that the “cause” variable be labeled on the X axis and the “effect” variable be labeled on the Y axis. – Plot the data pairs on the diagram. – Interpret the scatter diagram for direction and strength. 12/31/2024 CSE3423 98  Scatter Diagrams Slide 3 of 4 An Example of When a Scatter Diagram Can Be Used – A scatter diagram can be used to identify the relationship between the production speed of an operation and the number of defective parts made. 12/31/2024 CSE3423 99  Scatter Diagrams Slide 4 of 4 An Example of When a Scatter Diagram Can Be Used (cont.) – Displaying the direction of the relationship will determine whether increasing the assembly line speed will increase or decrease the number of defective parts made. Also, the strength of the relationship between the assembly line speed and the number of defective parts produced is determined. – Example obtained from: 12/31/2024 CSE3423 100  Flow Charts Slide 1 of 3 Flow Charts Defined – A flow chart is a pictorial representation showing all of the steps of a process. 12/31/2024 CSE3423 101  Flow Charts Slide 2 of 3 Creating a Flow Chart – First, familiarize the participants with the flow chart symbols. – Draw the process flow chart and fill it out in detail about each element. – Analyze the flow chart. Determine which steps add value and which don’t in the process of simplifying the work. 12/31/2024 CSE3423 102  Flow Charts Slide 3 of 3 Examples of When to Use a Flow Chart – Two separate stages of a process flow chart should be considered: The making of the product The finished product 12/31/2024 CSE3423 103  Run Charts Slide 1 of 3 Run Charts Defined – Run charts are used to analyze processes according to time or order. 12/31/2024 CSE3423 104  Run Charts Slide 2 of 3 Creating a Run Chart – Gathering Data Some type of process or operation must be available to take measurements for analysis. – Organizing Data Data must be divided into two sets of values X and Y. X values represent time and values of Y represent the measurements taken from the manufacturing process or operation. – Charting Data Plot the Y values versus the X values. – Interpreting Data Interpret the data and draw any conclusions that will be beneficial to the process or operation. 12/31/2024 CSE3423 105  Run Charts Slide 3 of 3 An Example of Using a Run Chart – An organization’s desire is to have their product arrive to their customers on time, but they have noticed that it doesn’t take the same amount of time each day of the week. They decided to monitor the amount of time it takes to deliver their product over the next few weeks. 12/31/2024 CSE3423 106  Control Charts Slide 1 of 3 Control Charts Defined – Control charts are used to determine whether a process will produce a product or service with consistent measurable properties. 12/31/2024 CSE3423 107  Control Charts Slide 2 of 3 Steps Used in Developing Process Control Charts – Identify critical operations in the process where inspection might be needed. – Identify critical product characteristics. – Determine whether the critical product characteristic is a variable or an attribute. – Select the appropriate process control chart. – Establish the control limits and use the chart to monitor and improve. – Update the limits. 12/31/2024 CSE3423 108  Control Charts Slide 3 of 3 An Example of When to Use a Control Chart – Counting the number of defective products or services Do you count the number of defects in a given product or service? Is the number of units checked or tested constant? 12/31/2024 CSE3423 109  Activity Process Flow Chart for Finding the Best Way Home – Construct a process flow chart by making the best decisions in finding the best route home. – Refer to the prior notes on flowcharts. Remember: Define and analyze the process, build a step-by step picture of the process, and define areas of improvement in the process. » Answer is on the next slide » Example obtained from: 12/31/2024 CSE3423 110 12/31/2024 CSE3423 111  Summary This presentation provided learning material for each of Ishikawa’s seven basic tools of quality. Each tool was clearly defined with definitions, a step-by-step process and an example of how the tool can be used. As seen through the presentation, these tools are rather simple and effective. 12/31/2024 CSE3423 112  Works - Cited Histograms and Bar Graphs. Your MBA: The Business Study Reference Site. http://yourmba.co.uk/pareto_diagram.htm Hci Home Services. Cause and Effect Diagram. http://hci.com.au/hcisite/toolkit/causeand.htm Scatter Diagram. http://sytsma.com/tqmtools/Scat.html Flowchart. Run Charts/Time Plot/ Trend Chart. Foster Thomas S. Managing Quality An Integrative Approach. New Jersey: Prentice Hall, 2001 12/31/2024 CSE3423 113 Topic : Software Quality Assurance Frameworks Ily Amalina Ahmad Sabri Aziz Deraman  SQA Definition  The function of software quality that assures that the standards, processes, and procedures are appropriate for the project and are correctly implemented (NASA) Objective: to minimise the cost of guaranteeing quality by a variety of activities performed throughout the development/manufacturing processes/stages 115 12/31/2024CSE3423  SQA vs Software Quality Control  SQC: a set of activities designed to evaluate the quality of a developed or manufactured product (IEEE, 1991)  Objective: holding back (for what?) any product that does not qualify (quality requirement!) 116 12/31/2024CSE3423  Software Quality Control (short notes)  Quality control should be the responsibility of the organizational unit producing the product.  It is possible to have the same group that builds the product perform the quality control function, or to establish a quality control group or department within the organizational unit that develops the product.  For software products, quality control typically includes specification reviews, inspections of code and documents, and checks for user deliverables. 117 CSE3423 12/31/2024  Software Quality Control (short notes)  Inspections  Usually, document and product inspections are conducted at each life cycle milestone to demonstrate that the items produced are within the criteria specified by the software quality assurance plan.  These criteria are normally provided in the requirements specifications, conceptual and detailed design documents, and test plans.  The documents given to users are the requirement specifications, design documentation, results from the user acceptance test, the software code, user guide, and the operations and maintenance guide.  Additional documents are specified in the software quality assurance plan. 118 CSE3423 12/31/2024  Software Quality Control (short notes)  Management  Quality control can be provided by various sources. For small projects, the project personnel’s peer group or the department’s software quality coordinator can inspect the documents.  On large projects, a configuration control board may be responsible for quality control. The board may include the users or a user representative, a member of the software quality assurance department, and the project leader. 119 CSE3423 12/31/2024  SQA Activities  The objectives of SQA activities refer to the functional, managerial and economic aspects of soft dev and soft maintenance  Soft dev (Process-oriented)  Soft maintenance (Product- 1. Assuring the soft will conform oriented) to functional tech req 1. Assuring the soft maint activities will conform to the funct tech req 2. Will conform to managerial 2. Will conform to managerial scheduling and budgetary req scheduling and budgetary req 3. Initiating and managing activities to improve and 3. Initiating and managing activities increase the efficiency of software maintenance and SQA activities to improve and increase the efficiency of software maintenance and SQA activities 120 CSE3423 12/31/2024 121 CSE3423 12/31/2024 122 CSE3423 12/31/2024  Software Architecture & Quality  The notion of quality is central in software architecting: a software architecture is devised to gain insight in the qualities of a system at the earliest possible stage.  Some qualities are observable via execution: performance, security, availability, functionality, usability  And some are not observable via execution: modifiability, portability, reusability, integrability, testability 123 CSE3423 12/31/2024 Pujangga Sains Komputer berkata: “To Measure the Unmeasurable …” (Tom Gilb) “You can’t control what you can’t measure…” (Tom DeMarco, 82) Ref: (Risk Assessment) 124 CSE3423 12/31/2024  SQA SYSTEM COMPONENTS CLASSES (Galin)  Pre-Project  Project life-cycle activities assessment  Infrastructure error prevention and improvement  Software quality management  Standardization, certiifcation dan SQA System  Organizing for SQA (the Human) 125 CSE3423 12/31/2024 Pre-Project  Objective: To improve the preparatory steps taken prior to initiating work on the project; a. Contract Review i. Clarification of the customer’s req ii. Review of project’s schedule and resource req estimates iii. Evaluation of the professional staff’s capacity iv. Evaluation of the customer’s capacity to fulfill his oblig v. Evaluation of the development risks 126 CSE3423 12/31/2024 Pre-Project  Objective: To improve the preparatory steps taken prior to initiating work on the project; b. Development i. Schedules ii. Req manpower and hardware resources iii. Risk evaluations iv. Org. issues: team members, subcontractors dan partnerships v. Project methodology & dev tools vi. Software reuse plan … and quality plans i. Quality goal (measurable terms) ii. Criteria for starting & ending each proj stage iii. Lists of rev, tests, and other schedule verification & validation activies 127 CSE3423 12/31/2024  Project life-cycle activities assessment (DLC & O-M)  Reviews – formal design reviews (DR) & peer reviews (various design deliverables)  Expert opinions – introducing additional external capablities  Software Testing – a formal SQA component that are targeted toward review of the actual running of the software  Software Maintenance – services to include Corrective, Adaptive and Preventive maintenance  Assurance of the quality of the external participant’s work – to establish effective controls over the external participant’s work. 128 CSE3423 12/31/2024  Infrastructure error prevention and improvement i. Procedures and work instructions (based on the organisational knowledge and experiences for effective performance ) ii. Supporting quality devices (ex: templates and checklists) iii. Staff training, instruction and certification (training new staffs, updating knowledge and experience of the staff and certifying quality staff) iv. Preventive and corrective actions (based on the professionally collected data during previous activities) v. Configuration management (establishing procedures to control the change process) vi. Documentation control (maintaining the availability of all important documents for the software) 129 CSE3423 12/31/2024  Software quality management  Objective: support the managerial control of software dev proj and maintenance services a. Project progress control (focus on resouces usage; schedules, risk management activities, & budget) b. Software quality metrics (quality of SD&M activities; Dev Team productivity; Help desk & Maintenace Team activities; Software fault density & schecdule deviations) c. Sofware quality costs (managing cost of SQA for Q software) 130 CSE3423 12/31/2024  Standardization, certification dan SQA System Obj: -utilization of international professional knowledge -improvement of coordination with other organization’s quality systems - professional evaluation a. Quality management standards i. SEI CMM assessement standard ii. ISO 9001 and ISO 9000-3 standards b. Project process standards i. IEEE 1012 standard ii. ISO/IEC 12207 standard 131 CSE3423 12/31/2024 Organizing for SQA (the Human) OBJ: - to dev and support impl of SQA comp. - to detect deviations from SQA Proc and Meth - to suggest improvements to SQA Comp a. The SQA unit b. SQA trustees, committees and forums 132 CSE3423 12/31/2024 Topic : Software Quality Assurance (Standard & Plan) Ily Amalina Ahmad Sabri Aziz Deraman Owner: Mark J. Tseytlin Sr. SQE, Raytheon Company What is Software Quality Assurance? 134 What is Quality? Quality – developed product meets it’s specification Problems: Development organization has requirements exceeding customer's specifications (added cost of product development) Certain quality characteristics can not be specified in unambiguous terms (i.e. maintainability) Even if the product conforms to it’s specifications, users may not consider it to be a quality product (because users may not be involved in the development of the requirements) 135 What is Quality Management? Quality Management – ensuring that required level of product quality is achieved Defining procedures and standards Applying procedures and standards to the product and process Checking that procedures are followed Collecting and analyzing various quality data Problems: Intangible aspects of software quality can’t be standardized (i.e elegance and readability) 136 What are SQA, SQP, SQC, and SQM? SQA includes all 4 elements… Software Quality Assurance – establishment of network of organizational procedures and standards leading to high- quality software 2. Software Quality Planning – selection of appropriate procedures and standards from this framework and adaptation of these to specific software project 3. Software Quality Control – definition and enactment of processes that ensure that project quality procedures and standards are being followed by the software development team 4. Software Quality Metrics – collecting and analyzing quality data to predict and control quality of the software product being developed 137 Software Development Process 138 Software Development Process Spiral Top-Down Design Model IPDS Software Development Phases 139 Software Development Cycle Software Development Phase End product Software requirements SRS, IRS Preliminary Design SDD, PQT/FAT/SAT Plans & Proc.’s, ICD, IDD Detailed Design PDL, User Manuals Code Code, UT Plan & Proc.’s Unit Test UT Results Software integration VDD Software Component Test PQT Report Software System Test FAT & SAT Reports Maintenance and Support ECP’s leading to updates 140 Software Development Cycle (contd.) Req. Maint PD DD SAT FAT CUT PQT Int 141 Integrated Product Development System 142 Software Quality Assurance Element I 143 Software Development Standards 144 Why are Standards Important? Standards provide encapsulation of best, or at least most appropriate, practice Standards provide a framework around which the quality assurance process may be implemented Standards assist in continuity of work when it’s carried out by different people throughout the software product lifecycle Standards should not be avoided. If they are too extensive for the task at hand, then they should be tailored. 145 SDS a Simplistic approach In most mature organizations: ISO is not the only source of SDS Process and Product standards are derived independently Product standards are not created by SQA 146 Process Standards Process Standards – standards that define the process which should be followed during software development ISO CMM CMMI Organizational Organizational IPDS Quality Manual SD Process STD’s Project Project SD Process STD’s Project SQP (SDP, IP, Method Sheets) SCMP 147 Product Standards Product Standards – standards that apply to software product being developed ISO MIL/ Industry STD’s STD’s Organizational Product STD’s COTS Project Product STD’s STD’s (SDP, IP, Method Sheets) 148 Quality Models 149 ISO - 9001 Elements Quality System Requirements Software Quality Responsibilities Management Responsibility Management Responsibility Quality system Quality system Contract review Contract review Design Control Design Control Document control Document control Purchasing Purchasing Purchaser supplied product - Product identification and traceability Product identification and traceability Process control Process control Inspection and testing Inspection and testing Inspection, measuring and test equipment - Inspection and test status Inspection and test status Control of non-conforming product - Corrective action Corrective action Handling, storage, preservation, packaging - and shipping Quality records Quality records Internal quality audits Internal quality audits Training Training Servicing - Statistical techniques Statistical techniques 150 Capability Maturity Model KPA’s 151 CMM Level’s 2-5 152 CMM Integration Model Architecture 153 CMM to CMMI Mapping 154 Documentation 155 Documentation Hierarchy Documents are not the only tangible way of representing software products. The working software system is the most tangible way of representing software products. Documents are the best way to ensure software products’ 156 understandability Process and Product Quality Quality of development process directly affects the quality of delivered products. This is the factory approach. It doesn’t work because software is designed rather then manufactured. 157 Process and Product Quality Creative Approach Quality Improvement – identifying good quality products, examining the processes used to develop these products, and then generalizing these processes so that they can be applied everywhere 158 Quality Improvement – The Wheel of 6Sigma Six Sigma 159 Quality Improvement – Six Sigma Process Visualize – Understand how it works now and imagine how it will work in the future Commit – Obtain commitment to change from the stakeholders Prioritize – Define priorities for incremental improvements Characterize – Define existing process and define the time progression for incremental improvements Improve – Design and implement identified improvements Achieve – Realize the results of the change 160 Continuity and Independence of SQA Software Quality Assurance team must be independent in order to take an objective view of the process and report problems to senior management directly If prescribed process is inappropriate for the type of software product which is being developed, then it should be tailored The standards must be upheld no matter how small the task. Prototyping doesn’t mean no standards. It means tailored standards. Quality is FREE, if it’s Everyone’s Responsibility! 161 Software Quality Planning Element II 162 Software Quality Plan Tailoring - SQP should select those organizational standards that are appropriate to a particular product Standardization - SQP should use (call out) only approved organizational process and product standards If new standards are required a quality improvement should be initiated Elements - SQP elements are usually based on the ISO-9001 model elements SQP is not written for software developers. It’s written for SQE’s as a guide for SQC and for the customer to monitor development activities Things like software production, software product plans and risk management should be defined in SDP, IP Quality Factor’s shouldn’t be sacrificed to achieve efficiency. Don’t take the job if quality process can’t be upheld 163 Software Quality Control Element III 164 Methods of Software Quality Control SQC involves overseeing the software development process to ensure that the procedures and STD’s are being followed The following activities constitute SQC: Quality Reviews - in-process reviews of processes and products Reviews are the most widely used method of validating the quality of processes and products. Reviews make quality everyone's responsibility. Quality must be built-in. SQE is responsible for writing Quality Engineering Records (QERs) documenting their participation in these reviews. Tests - end-result verifications of products. These verifications are conducted after the software has been developed. Test procedures are followed during conduct of these activities. SQE is responsible for keeping the logs and some times for writing the test report. Quality Audits - in-process verifications of processes. These audits are conducted periodically (twice a month) to assess compliance to the process STD’s. 165 Quality Reviews Peer reviews - reviews of processes and products by groups of people. These reviews require pre-review preparation by all participants. If a participant is not prepared, then the review is not effective. This type of review requires participation of the SQE, moderator, recorder, author(s), and one or more critical reviewers. All issues found during these reviews are documented on AR forms. Walkthroughs - reviews of products by groups of people mostly without preparation. For example a requirements traceability review is a walkthrough. It involves tracing a requirement from customer requirements to the test procedures. All issues found during these reviews are documented on CAR forms. Desk inspections - reviews of products by individuals. These reviews involve people reviewing products by themselves (not in a group) and then submitting their comments to the author(s). The issues found during these reviews are treated in informal manner. 166 Tests Engineering Dry-run - test conducted by engineering without SQE. These tests include Unit Tests and engineering dry-runs of the formal tests. These engineering dry- runs are used to verify correctness and completeness of the test procedures. Also, these is the final engineering verification of the end-product before sell-off to SQE. All issues found during these tests are documented on STR forms. SQE Dry-run - test conducted by SQE. These tests include PQT, FAT and SAT dry- runs. These tests are used to verify the end-product before the formal test with the customer. An SQE is sometimes responsible for writing the test report. However, if a separate test group is available, then SQE is relived of this obligation. All issues found during these tests are documented on STR forms. TFR - test conducted as “RFR - run-for-record” with the SQE and the customer. These tests include FAT and SAT. These tests are conducted to sell the end-product off to the customer. SQE is present at all such tests. All issues found during these tests are documented on STR forms. 167 Quality Audits SQE Audits - audits conducted by SQE to verify that the process STD’s are being followed. Examples of these audits are IPDS compliance, Configuration Control, and Software Engineering Management. All findings for these audits are documented on QER forms. The results of the audits are distributed to the next level of management (above project level). If the issue(s) are not fixed then the findings are elevated to upper management. Independent Audits - audits conducted by ISO generalists or other independent entities to verify that the process STD’s are being followed. These audits are usually conducted on a division/facility level. The results of these audits are distributed to upper management. 168 Defect Detection Formal bug finding activities include Quality Reviews and Tests s i s ys i ys al al st An An Te ts re ts en n tu tio en n m ap gn io ca m re at C i re n ifi es