CSE241/CMM341 Software Quality Foundations Of Software Engineering PDF

CPT 441 CSE241/CMM341 Software Quality Foundations Assurance of Software & Testing Engineering Photo credit: Dreamstime.com Topic: Quality Management Topic 12. 1. Software quality ma...

CPT 441 CSE241/CMM341 Software Quality Foundations Assurance of Software & Testing Engineering Photo credit: Dreamstime.com Topic: Quality Management Topic 12. 1. Software quality management Quality Management 2. Software quality 3. Software measurement 2  Learning Outcomes the quality management process and know why quality Understand planning is important the concept of software quality and quality Understand characteristics according to ISO 9126 how measurement may be helpful in assessing some Understand software quality attributes 3 4  Software quality management (SQM) Software quality management (SQM) is concerned with ensuring the developed software systems are “fit for purpose” What does it mean? Systems should meet the needs of their users, should perform efficiently and reliably, and should be delivered on time and within budget. 5 Software fitness for purpose – Questions to ask Has the software been properly tested? Is the software sufficiently dependable to be put into use? Is the performance of the software acceptable for normal use? Is the software usable? Is the software well-structured and understandable? Have programming and documentation standards been followed in the development process? 10/12/2014 Chapter 24 Quality management 6 Scope of quality management For large, complex systems – very important The quality documentation is a record of progress and supports continuity of development as the development team changes. For smaller systems quality management needs less documentation should focus on establishing a quality culture. Techniques have to evolve when agile development is used. 7 Process-based quality This figure illustrates the process-based approach to achieving product quality. However, there is a very complex and poorly understood relationship between software processes and product quality. 8 Process-based quality Software design is a creative process, so the influence of individual skills and experience is significant- particularly important in software development. External factors, such as the novelty of an application or commercial pressure for an early product release, also affect product quality irrespective of the process used. 9 Software quality management Software Software Quality Quality Planning Assurance (SQA Plan) (SQA) Software Software Process Quality Improvement Control (SQC) (SPI) 10  Software quality assurance (SQA) SQA is not testing. SQA is a set of activities that define and assess the adequacy of software processes to provide evidence that establishes confidence that the software processes are appropriate and produce software products of suitable quality for their intended purposes. Quality assurance is concerned with both the software product and the development process. 11  Software process quality The SE process influences the quality characteristics of software products, which in turn affect quality as perceived by stakeholders. Software product quality Sound engineering practice requires that intermediate work-products relevant to quality be evaluated throughout the software engineering process. 12 SQA Plan The SQA plan identifies: documents, standards, practices, and conventions governing the project and how they will be checked and monitored to ensure adequacy and compliance also measures, statistical techniques, procedures for problem reporting and corrective action, resources such as tools, techniques and methodologies, training and SQA reporting and documentation 13 TASKS OF SQA SQA TASKS ANALYTICAL CONSTRUCTIVE  Tasks of evaluating and  Tasks of applying verifying the conformity technical and of the software product organizational means to the requirements. during development to ensure that the requirements are satisfied 14 Shared good practice for Quality Management Check before check-in Programmers are responsible for organizing their own code reviews with other team members before the code is checked in to the build system. Never break the build Team members should not check in code that causes the system to fail. Developers have to test their code changes against the whole system and be confident that these work as expected. Fix problems when you see them If a programmer discovers problems or obscurities in code developed by someone else, they can fix these directly rather than referring them back to the original developer. 10/12/2014 Chapter 24 Quality management 15 Pair programming This is an approach where 2 people are responsible for code development and work together to achieve this. Code developed by an individual is therefore constantly being examined and reviewed by another team member. https://railsware.com/blog/what-is-pair-programming/ 10/12/2014 Chapter 24 Quality management 16  Pair programming Pair programming leads to a deep knowledge of a program, as both programmers have to understand the program in detail to continue development. This depth of knowledge is difficult to achieve in inspection processes and pair programming can find bugs that would not be discovered in formal inspections. https://railsware.com/blog/what-is-pair-programming/ 10/12/2014 Chapter 24 Quality management 17 Pair programming weaknesses Mutual misunderstandings Both members of a pair may make the same mistake in understanding the system requirements. Discussions may reinforce these errors. Pair reputation Pairs may be reluctant to look for errors because they do not want to slow down the progress of the project. Working relationships The pair’s ability to discover defects is likely to be compromised by their close working relationship that often leads to reluctance to criticize work partners. 10/12/2014 Chapter 24 Quality management 18 Class Activity 1 – Apps Quality Review Work in Project Work Go to Google Playstore / Apps Store, identify 1 quality concern (bug, error, performance issue) of the following apps by referring to the rating & review comments: Group 1-15: CIMB Clicks Malaysia Group 16-30: Domino's Pizza Malaysia Group 31-45: Waze Group 46-56: Shopee Provide your group number + answer as new post. [No submission, no attendance!] Deadline: 10 January 11.59pm https://csusm.padlet.org/cylim/cse241cmm341 19  Part 2 Software Quality 20  For a software application to achieve high quality levels, it is necessary to begin upstream and ensure that intermediate deliverables and work products are also of high-quality levels. This means that the entire process of software development must itself be focused on quality Achieving (Caper Jones) software quality 21 The The degree to which a system, component, or process meets specified requirements, and customer/user needs or definitions expectations of software [IEEE 610] quality A set of characteristics of a software product by which its quality is described and evaluated. A software quality characteristic may be refined into multiple levels of sub-characteristics [ISO/IEC 9126] ISO/IEC 9126 is used as a guide to describe software quality characteristics 22 ISO 9001 standards framework An international set of standards that can be used as a basis for developing quality management systems. ISO 9001, the most general of these standards, applies to organizations that design, develop and maintain products, including software. The ISO 9001 standard is a framework for developing software standards. It sets out general quality principles, describes quality processes in general and lays out the organizational standards and procedures that should be defined. These should be documented in an organizational quality manual. 10/12/2014 Chapter 24 Quality management 23 ISO/IEC 9126 – Characteristics Internal: static properties of the code Quality in use: the extent to which the software meets the needs of the user External: behaviour of the software when it’s executed 24 ISO/IEC 9126 – quality in use 25 external & internal quality: functionality Existence of functions with specified attributes. These functions meet the specified or implied requirements 26 external & internal quality: reliability Attributes that relate to the ability of the software to maintain a specified level of performance under given conditions and for a specified time period. 27 external & internal quality: usability Attributes that relate to the effort required to use, and on the individual assessment of such use by a specific or implied group of users. 28 external & internal quality: efficiency Attributes that relate to the relationship between the performance levels of the software and the amount of equipment used under specified conditions. 29 external & internal quality: maintainability Attributes that relate to the effort that is necessary to carry out software changes. 30 external & internal quality: portability Attributes that relate to the ability of software to be transferred from one environment to another. 31 ISO/IEC 9126: Recap (1/2) 32  ISO/IEC 9126: Recap (2/2) The definitions emphasize that quality is dependent upon users’/customers’ requirements. Reaching agreement on what constitutes quality for all stakeholders – require quality to be formally defined/discussed. Scientific research has shown that the entire process of software development must itself be focused on quality. 33 Class Activity 2 – Apps Quality Review Go back to your answer in activity 1, use “Add comment” to add in the relevant quality attributes for the error/bug/ performance issue which you have provided. https://csusm.padlet.org/cylim/cse241cmm341 34 Part 3 Software Measurement 35  Software measurement is concerned with deriving a numeric value for an attribute of a software product or process. This allows for objective comparisons between techniques and processes. Software measurement Most organisations still don’t make systematic use of software measurement. There are few established standards in this area. 36  A software metric is a characteristic of: a software system, a development process, or a related documentation or that can be objectively measured. E.g. Lines of code in a program, number of person-days required to develop a component, time spent on the process by particular engineers, Software Allow the software and the software process to be metric quantified. May be used to predict product attributes or to control the software process. Product metrics can be used for general predictions or to identify anomalous components. 37  Use of measurements To assign a value to system To identify the system components whose quality attributes quality is sub-standard By measuring the characteristics of Measurements can identify system components, and then individual components with aggregating these measurements, characteristics that deviate from you can assess system quality the norm. attributes, such as maintainability. 38  A software property can be measured accurately. The relationship exists between what Metrics we can measure and what we want assumptions to know. This relationship has been formalised and validated. 39  Problems with measurement in industry It is impossible to quantify the return on investment of introducing an organizational metrics program. There are no standards for software metrics or standardized processes for measurement and analysis. In many companies, software processes are not standardized and are poorly defined and controlled. Most software measurement focused on code-based metrics and plan-driven development processes. However, more software is now developed by configuring ERP systems or Commercial off-the-shelf (COTS). Introducing measurement adds additional overhead to processes. 40 Product metrics A quality metric should be a predictor of product quality. Classes of product metric Dynamic metrics Static metrics Collected by measurements Collected by measurements made of a program in made of the system execution; representations; Help to assess efficiency and Help to assess complexity, reliability understandability and maintainability. 10/12/2014 Chapter 24 Quality management 41 Dynamic and static metrics Dynamic metrics are closely related to software quality attributes Static metrics have an indirect It is relatively easy to measure the response time of a system (performance relationship with quality attributes attribute) or the number of failures You need to try and derive a relationship (reliability attribute). between these metrics and properties such as complexity, understandability and maintainability. 10/12/2014 Chapter 24 Quality management 42 Metrics for monitoring testing 43 Test Metrics Defect-based metrics e.g. number of detected defects Test case-based metrics e.g. number of planned test cases Test object-based metrics e.g code coverage (statements, decisions, paths) 44 Common defect-based metrics 45  Quantity of executed/not executed test cases Common Completion of test case preparation (%) Test Case- Based Completion of preparing the test environment (%) Metrics Test case execution (e.g. number of test cases run/not run, and number of test cases passed/failed) 46  The extent of test coverage achieved Requirements, code, risks, etc including pass/fail results Common Test Object- Other metrics to aid monitoring: Based level of confidence of testers in the Metrics product- subjective dates of milestones testing costs, including the cost of finding the next defect or running the next test compared to the benefit 47  Example of testing metrics https://www.softwaretestinghelp.com/software-test-metrics- and-measurements/ More examples of test metrics 49  When you collect quantitative data about software and software processes, you have to analyze that data to understand its meaning. It is easy to misinterpret data and to make Measurement inferences that are incorrect. ambiguity Must also consider the context where the data is collected, not just data on its own 50 Software Analytics Software analytics is analytics on software data for managers and software engineers with the aim of empowering software development individuals and teams to gain and share insight from their data to make better decisions. 10/12/2014 Chapter 24 Quality management 51 Software Analytics Enablers The automated collection of user data by software product companies when their product is used. If the software fails, information about the failure and the state of the system can be sent over the Internet from the user’s computer to servers run by the product developer. The use of open source software available on platforms such as Sourceforge and GitHub and open source repositories of software engineering data. The source code of open source software is available for automated analysis and this can sometimes be linked with data in the open source repository. 10/12/2014 Chapter 24 Quality management 52 Analytics tool use Tools should be easy to use as managers are unlikely to have experience with analysis. Tools should run quickly and produce concise outputs rather than large volumes of information. Tools should make many measurements using as many parameters as possible. It is impossible to predict in advance what insights might emerge. Tools should be interactive and allow managers and developers to explore the analyses. 10/12/2014 Chapter 24 Quality management 53 Status of software analytics Software analytics is still immature and it is too early to say what effect it will have. Not only are there general problems of ‘big data’ processing, our knowledge depends on collected data from large companies. This is primarily from software products and it is unclear if the tools and techniques that are appropriate for products can also be used with custom software. Small companies are unlikely to invest in the data collection systems that are required for automated analysis so may not be able to use software analytics. 10/12/2014 Chapter 24 Quality management 54  Key points (1/2) ▪ Software quality management is concerned with ensuring that software has a low number of defects and that it reaches the required standards of maintainability, reliability, portability etc. ▪ Software standards are important for quality assurance as they represent an identification of ‘best practice’. ▪ When developing software, standards provide a solid foundation for building good quality software. ▪ Reviews of the software process deliverables involve a team of people who check that quality standards are being followed. Reviews are the most widely used technique for assessing quality. 55  Key points (2/2) ▪ You may be able to use the values of the software metrics that are collected to make inferences about product and process quality. ▪ Product quality metrics are particularly useful for highlighting anomalous components that may have quality problems. These components should then be analyzed in more detail. ▪ Software analytics is the automated analysis of large volumes of software product and process data to discover relationships that may provide insights for project managers and developers. 56  Main references Please read the book slides for details of quality measurement and metrics. Ian Sommerville, Software Engineering, 10th Edition, Pearson, 2016. Other references: ▪ Khsirasagar, N. & Priyadarshi, T., Software Testing and Quality Assurance, John Wiley & Sons Publication, 2008. ▪ Black, R., Veenendaal, E. V. & Graham D., Foundations of Software Testing, ISTQB Certification, 3rd Edition, Cengage Learning, 2012. 57 58