Chapter 5 Reviews.pdf

953422 Software Quality Assurance Chapter 5 Reviews 1/2024 Lecturer Kittitouch Suteeca  Chapter 5 Review Software Quality Assurance Claude Y Laporte and Alain April Department of Software and IT Engineering...

953422 Software Quality Assurance Chapter 5 Reviews 1/2024 Lecturer Kittitouch Suteeca  Chapter 5 Review Software Quality Assurance Claude Y Laporte and Alain April Department of Software and IT Engineering Chapter 5 - Reviews “A year of work experience in an environment that requires regular participation in peer reviews is the equivalent of 3 years of work experience in an environment where you do not use these reviews.” Gerald Weinberg, SQTE Magazine, March 2003 Software Quality Assurance Claude Y Laporte and Alain April Chapter 5 - Reviews This chapter presents different types of software reviews: personal review, the “desk check”, the walk-through and the inspection. We describe the theory about reviews and then practical examples. It introduces reviews in an agile context. Subsequently, we describe other reviews specific to a project: the project launch review and lessons learned review. The chapter concludes with a discussion on the selection of one type of review depending on your business domain and how these techniques fit into the software quality assurance plan. 4 Chapter 5 - Objectives After studying this chapter, you will be able to: understand the value of different types of reviews; understand the personal review; understand the desk-check type of peer review; understand the reviews described in the ISO/IEC 20246 standard, the CMMI® and the IEEE 1028 standard; understand the walk-through and inspection review; understand the project launch review and project lessons learned review; understand the measures related to reviews; understand the usefulness of reviews for different business models; understand the requirements of the IEEE 730 standard regarding reviews. 5 Characteristics of Informal Reviews There is no documented process to describe reviews and they are carried out in many ways by different people in the organization; Participants’ roles are not defined; Reviews have no objective, such as fault detection rate; They are not planned, they are improvised; Measures, such as the number of defects, are not collected; The effectiveness of reviews is not monitored by management; There is no standard that describes them; No checklist is used to identify defects. 6  Review A process or meeting during which a work product, or set of work products, is presented to project personnel, managers, users, customers, or other interested parties for comment or approval. ISO 24765 A process or meeting during which a software product, set of software products, or a software process is presented to project personnel, managers, users, customers, user representatives, auditors or other interested parties for examination, comment or approval. IEEE 1028 7 Types of reviews used during the software development cycle System System System Quality Specification Assurance Plan Integration and and Design Validation Software Production Requirement VALIDATION Specification Preliminary Design Programming Detailed Design, Coding, Unit testing, integration Internal quality control actions Project launch End of Phase End of Phase End of Phase End of Project Review Review Review Review Review SQAP writing Document reviews Inspections Metrology Audits Project Reviews 8 © 1990 - ALSTOM Transport SA Objectives of a Review 1. Estimate/measure quality 2. Identify defects for removal 3. Reduce cost and time of future document production (i.e. learning process) 4. Get management to take responsibility for resource approval decisions 5. Get a technological group to take responsibility for technical decisions 6. Teach writers how to follow technical standards 7. Motivate writers to follow technical standards 8. Estimate impacts (cost) of continuing with current plans (delays, rework, maintenance and defects removal costs at later stages) 9. Estimate the remaining major defects, with or without removal of the ones found in the review. 10. Stimulate the creation and contribution of better ideas, in the review or later 11. Measure the efficiency and effectiveness of processes 12. Measure the productivity and quality of organizations, teams and individuals. 13. Reduce defect content by removal of defects identified 14. Reduce the costs of testing 15. Reduce the time to market 16. Give early feedback, before too much is invested in some work 17. Determine entry criteria to a process 18. Determine exit criteria from a process 9 Process of developing a document Source Document(s) Document Software development Product Standard(s) activities and template(s) 10 Review Process Software Product to List of anomalies Review and decisions taken Standard(s) and template(s) Measure(s) Review Activities Change Checklist(s) Request(s) (optional) Corrected Source Software Document(s) Product 11  Error detection during the software development life cycle - 1 Injected Detected without Inspections New Gap Detected with Inspections 25 20 Defects/KLOC Gap 15 10 5 0 START REQ HLD LLD CODE UT IT ST SHIP Activity Assumption: Defect removal effectiveness for each test, inspection is 50% 12 Radice, ‘High Quality Low Cost Software Inspections’, 2002, chapter 1. Personal Review A personal review is done by the person reviewing his own software product in order to find and fix the most defects possible. A personal review should precede any activity that uses the software product under review. The principles of a personal review are: Find and correct all defects in the software product; Use a checklist produced from your personal data, if possible, using the type of defects that you are already aware of; Follow a structured review process; Use measures in your review; Use data to improve your review; Use data to determine where and why defects were introduced and then change your process to prevent similar defects in the future. 13 Personal Review The following practices should be followed to develop an effective and efficient personal review: Pause between the development of a software product and its review; Examine products in hard copy rather than electronically; Check each item on the checklist once completed; Update the checklists periodically to adjust to your personal data; Build and use a different checklist for each software product; Verify complex or critical elements with an in depth analysis. 14 Desk-Check Review Not described in standards Peer Desk Check (ISO 20246) Informal review where the author and a colleague walk through a work product Sometimes called the ‘Pass around’ (Wiegers) This type of peer review because it is inexpensive and easy to implement Used for low-risk software products According to Wiegers, this review is less intimidating than a group review such as a walk-through or inspection Wiegers is the author of a book about Peer Reviews ‘Peer Reviews in Software - A Practical Guide’ 15  Desk-Check Review ENTRY CRITERIA The document is ready for a review INPUT Software product to review Product..... document DC 100 Checklist Plan the Desk Check (optional) DC 150 Complete the Review Form DC 110 DC 120 DC 130 DC 140 Send Review Conduct Edit documents the meeting document. to reviewers product (if needed) document 16 A  Desk Check Review Form Desk check Review Form Name of author: _____________________________________________ Review Date (yyyy-mm-dd):_______________________ Document Title:______________________________________________ Reviewer Name: ________________________________ Document Version:________________________________ Comment Document Line # / Disposition of No. Page Location Comments comments * Remarks 1 2 3 4 5 6 7 Disposition of comment: Inc: Incorporate as is; NOT: Not incorporate, MOD: Incorporate with modification Effort to review document (hour): ________ Effort to correct document (hour): ________ Signature of Reviewer: ______________________ Signature of Author: ________________________ 17  Work product Artefact produced by a process. EXAMPLE: Project plan, requirements specification, design documentation, source code, test plan, test meeting minutes, schedules, budgets, and incident reports. Note 1 to entry: A subset of the work products will be baselined to be used as the basis of further work and some will form the set of project deliverables. ISO/IEC 20246 18  Anomaly Any condition that deviates from expectations based on requirements specifications, design documents, user documents, standards, etc., or from someone’s perceptions or experiences. NOTE—Anomalies may be found during, but not limited to, the review, test, analysis, compilation, or use of software products or applicable documentation. IEEE 1028 19 Types of Reviews and Audits - 1 Management review A systematic evaluation of a software product or process performed by or on behalf of the management to monitors progress, determines the status of plans and schedules, confirms requirements and their system allocation, or evaluates the effectiveness of the management approaches used to achieve fitness for purpose; Technical review A systematic evaluation of a software product by a team of qualified personnel that examine the suitability of the software product for its intended use and identify discrepancies from specifications and standards; Inspection A visual examination of a software product to detect and identify software anomalies including errors and deviations from standards and specifications; 20 Types of Reviews and Audits - 2 Walk-through A static analysis technique in which a designer or programmer leads members of the development team and other interested parties through a software product, and the participants ask questions and make comments about any anomalies, violation of development standards, and other problems; Audit An independent assessment, by a third party, of a software product, a process or a set of software processes to determine compliance with the specifications, standards, contractual agreements or other criteria. 21 Characteristics of reviews and audits in the IEEE 1028 Management Technical Inspection Walk-through Audit review Review Objective Monitor progress Evaluate Find anomalies; Find anomalies, Independently conformance to verify resolution; examine evaluate specifications and verify product alternatives; conformance with plans quality improve product; objective standards forum for learning and regulations Recommended Two or more people Two or more 3-6 2-7 1-5 group size people Volume of Moderate to High Moderate to High Relatively low Relatively low Moderate to High material Leadership Usually the Usually the lead Trained Facilitator or Lead auditor responsible manager engineer facilitator author Management Yes When No No No; however participates management management may be evidence or called upon to resolution may be provide evidence required Output Management review Technical review Anomaly list, Anomaly list, action Formal audit report; documentation documentation anomaly items, decisions, observations, summary, follow-up proposals findings, deficiencies inspection documentation 22 Walk-through “The purpose of a walk-through is to evaluate a software product. A walk-through can also be performed to create discussion for a software product” The main objectives of the walk-through are: find anomalies; improve the software product; consider alternative implementations; evaluate conformance to standards and specifications; evaluate the usability and accessibility of the software product. Reasons for the implementation of a walk-through process: identify errors to reduce their impact and the cost of correction; improve the development process; improve the quality of the software product; reduce development costs; reduce maintenance costs. 23  Walk-through Product..... document Change requests WT 100 Checklists Process Plan the Walk- improvements Request through Source WT 150 Complete Follow-up Exit & WT 110 WT 120 WT 130 WT 140 Release Conduct Conduct Conduct Edit Kickoff. Document Logging Document Meeting Checking Meeting IEEE 1028 also describes the procedures of walk-throughs 24 Identification of Roles and Responsibilities - 1 Walk-through leader conduct the walk-through; handle the administrative tasks pertaining to the walk-through (such as distributing documents and arranging the meeting); prepare the statement of objectives to guide the team through the walk- through; ensure that the team arrives at a decision or identified action for each discussion item; issue the walk-through output. Recorder note all decisions and identified actions arising during the walk-through meeting; note all comments made during the walk-through that pertain to anomalies found, questions of style, omissions, contradictions, suggestions for improvement, or alternative approaches. 25 Identification of Roles and Responsibilities - 2 Author present the software product in the walk-through. Team member adequately prepare for and actively participate in the walk-through; identify and describe anomalies in the software product. 26 Inspection Review Inspection process that Michael Fagan developed at IBM in the 1970s to increase the quality and productivity of software development. Purpose of the inspection To detect and identify anomalies of a software product including errors and deviations from standards and specifications (IEEE 1028) It is more economical and efficient to detect and correct errors as soon as possible. Inspection is a very effective method to detect these errors or anomalies. 27 Use of Inspection a) Verify that the software product satisfies its specifications; b) Check that the software product exhibits the specified quality attributes; c) Verify that the software product conforms to applicable regulations, standards, guidelines, plans, specifications and procedures; d) Identify deviations from provisions of item a), item b), and c) e) Collect data, for example, the details of each anomaly and effort associated with their identification and correction. f) Request or grant waivers for violation of standards where the adjudication of the type and extent of violations are assigned to the inspection jurisdiction g) Use the data as input to project management decisions as appropriate (e.g. to make trade-offs between additional inspections versus additional testing). 28 Inspection Product..... document Change requests IP 100 Process Checklists Plan improvements Inspection Inspection Rules Source IP 150 Request Complete Follow-up Exit & IP 110 IP 120 IP 130 IP 140 Release Conduct Conduct Conduct Edit. Kickoff Document Logging Document Meeting Checking Meeting IEEE 1028 also describes the procedures of inspection 29  Rate Guidelines After Many Months of Utilisation Type of document Inspected Inspection rate Architecture 2 PPH to 3 PPH (see NOTE 1) Requirements 2 PPH to 3 PPH Preliminary design 3 PPH to 4 PPH Detailed design 3 PPH to 4 PPH Source code 100 LPH to 200 LPH (see NOTE 2) Test plan 5 PPH to 7 PPH Fixes and changes 50 LPH to 75 LPH User documentation 8 PPH to 20 PPH NOTE 1 – Page per hour. NOTE 2 – Lines (of code) per hour Sources : Radice, ‘High Quality Low Cost Software Inspections’, 2002. IEEE Standard 1028-2008 – Guidelines for typical inspection rates and anomaly recording rates 30 Project Launch review The project launch review is a management review of: Milestone dates, requirements, schedule, budget constraints, deliverables, members of the development team, suppliers, etc. Some organizations also conduct kick-off reviews at the beginning of each of the major phases of the project when projects are spread over a long period of time (as in several years). 31 Project Launch review Before the start of a project, team members ask themselves the following questions: Who will the members of my team be? Who will be the team leader? What will my role and responsibilities be? What are the roles of the other team members and their responsibilities? Do the members of my team have all the skills and knowledge to work on this project? 32 Project Retrospectives The project retrospective review is normally carried out at the end of a project or at the end of a phase of a large project. We want to know what has been done well in this project, what has gone less well and what could be improved for the next project. Synonyms: lessons learned post mortem after-action-review 33  Post mortem A collective learning activity that can be organized for projects or at the end of a phase or at project completion. The main motivation is to reflect on what has happened in the project to improve future practices for individuals who participated in the project and for the organization as a whole. The result of a post-mortem is a report. [DIN 05] Lessons learned The knowledge gained during a project which shows how project events were addressed or should be addressed in the future with the purpose of improving future performance. PMBOK® Guide 34 Project Retrospectives Provides valuable information such as: updating project data such as length, size, defects and schedule; updating quality or performance data; a review of performance against plan; updating databases for size and productivity; adjustment of processes (e.g. checklist), if necessary, based on the data (notes taken on the proposal process improvement (PIP) forms, changes in design or code, lists of default controls indicated and so on). 35 Project Retrospectives Main items on the agenda during a project retrospective review are: list the major incidents and identify the main causes; list the actual costs and the actual time required for the project, and analyze variances from estimates; review the quality of processes, methods and tools used for the project; make proposals for future projects (e.g. indicate what to repeat or re-use (methodology, tools, etc.), what to improve and what to give up for future projects. 36 A Project Retrospectives In many organizations, the transfer of knowledge and lessons learned is not necessarily done from one project to another. For example, one of the co-authors conducted lessons learned sessions in a division of an international transport company. Several times during the same lessons learned session, participants raised problems encountered during the project that had just come to an end and other participants said they had the same problems in previous projects! 37 A