Quality Management PDF

Quality Management Goals of the Unit Understand the importance of quality management in software development projects Learn the main techniques to manage quality in projects Learn the main techniques to manage quality of project deliverables Understand the...

Quality Management Goals of the Unit Understand the importance of quality management in software development projects Learn the main techniques to manage quality in projects Learn the main techniques to manage quality of project deliverables Understand the differences between software testing and quality management spm - ©2014 adolfo villafiorita - introduction to software project management 2 Project Quality Management Software Quality Assurance Software Quality Assurance Software quality assurance is the planned and systematic application of activities to ensure conformance of software life cycle processes and products to requirements, standards, and procedures spm - ©2014 adolfo villafiorita - introduction to software project management 6 Comments The definition applies both to the process and the products Quality assurance (like many other activities) is planned and systematic Conformance is required w.r.t. all the elements characterizing the software operational environment spm - ©2014 adolfo villafiorita - introduction to software project management 7 Quality Assurance Process Quality planning, which identifies the relevant standard and practices and the way to implement them Quality assurance, which focuses on ensuring that the project applies and follows the quality standards identified at the previous step Quality control, which ensures that the products respect the quality standards identified during the planning phase spm - ©2014 adolfo villafiorita - introduction to software project management 8 Quality Planning Plan Quality Management Quality Planning Goal: ensure the goals of quality management are met in a project Means: – Identification of constraints and quality goals in scope – Identification of standards and procedures to be applied – Identification of techniques to be applied – Allocation of resources (time, people, budget) to quality assurance activities – Roles and responsibilities Output: quality assurance planning document spm - ©2014 adolfo villafiorita - introduction to software project management 11 Comments Quality needs to be balanced with the other project constraints (e.g. time and costs) Not all systems are equally critical: NASA, for instance distinguishes eight different classes of software systems The quality assurance team should be independent from the development team Different “levels” of independence: – different roles in the project – different structures in the organization – independent organizations spm - ©2014 adolfo villafiorita - introduction to software project management 12 Quality Assurance & Quality Control Quality Assurance Goal: ensure that the project applies and follows the quality standards Main tool: quality audits Triggers: time, milestones, or critical events in the project (according to the quality plan) Quality audits include – Inspections – Reviews – Walkthroughs Output: – Main findings and recommendations spm - ©2014 adolfo villafiorita - introduction to software project management 16 Audit Meeting Organization Audit and review meetings are held to assess the status of a product or project Three “conflicting” roles: – The auditors: analysis of products and project documentation – The project members, responsible of providing clarifications and explain choices and project status – The moderator, who ensures the agenda is followed and the meeting environment remains productive spm - ©2014 adolfo villafiorita - introduction to software project management 17 Auditing Process Structure Definition of the goals and boundaries of the audit Identification of the auditing committee (independence, competence, professionalism) Distribution of all the relevant material Preparation of the auditing by the auditors Auditing meeting Preparation of the final report spm - ©2014 adolfo villafiorita - introduction to software project management 18 Signs of Troublesome Projects According to NASA signs of troublesome projects include: – Frequent changes in milestones – Unexplained fluctuations in personnel – Continued delays in software delivery – Unreasonable number of non conformance reports or change requests. spm - ©2014 adolfo villafiorita - introduction to software project management 19 Quality Control Quality Control Goal: ensure that the the products respect the quality standards identified during the planning phase Main tools: – Inspections – Analyses – Testing Triggers: milestones or critical events in the project (according to the quality plan) Output: – List of non-conformance reports spm - ©2014 adolfo villafiorita - introduction to software project management 21 Quality Control Quality control of software systems is extremely difficult, because: – of the enormous number of states a software system can be in (exhaustive testing is impractical/impossible) – the operating environment is unpredictable – discontinuity: little changes in inputs can cause enormous changes in outputs – non functional requirements can be difficult to assess (consider, e.g., maintainability, usability) – test automation can be difficult or very costly (consider, e.g., testing a GUI) – today’s systems are composed by using different technologies (e.g., HTML/CSS, Javascript, PHP, WebServer, OS) spm - ©2014 adolfo villafiorita - introduction to software project management 22 Quality Control Techniques for Software Walkthroughs and code inspections: – the system is analyzed by an independent team Analyses: – static checkers verify the correct use of certain syntactic constructs (e.g., no assignments in conditions) – dynamic checkers verify anomalies and suspect situations by executing instrumented code – code metrics are collected to measure other quality characteristics (e.g., comments/lines; unit test coverage) – formal verification (theorem proving, model checking) proves properties about abstract representations of the system Testing: – tests are performed on a system to verify the behavior under specific circumstances spm - ©2014 adolfo villafiorita - introduction to software project management 23 Establishing a Metrics Program Establishing a Metrics Program A metrics collection program quantitatively assesses how the project goals are being achieved Process metrics: measure different characteristics of a project Product metrics: measure different characteristics of a project product Trends often more important than point-wise numbers Better if automated spm - ©2014 adolfo villafiorita - introduction to software project management 25 Product Metrics: Size Size oriented metrics: – Source lines of code – Number of classes Function oriented metrics: – Function Points – Object Points Comments: – Size metrics can be automatically collected – The count of SLOC, however, is “controversial” (see next slide) – Function oriented metrics requires trained personnel for their collection spm - ©2014 adolfo villafiorita - introduction to software project management 26 Product Metrics: Complexity Cyclomatic complexity Coupling between objects Depth of inheritance Fan-in, fan-out spm - ©2014 adolfo villafiorita - introduction to software project management 27 Product Metrics: Quality Metrics Ratio between lines of comments and lines of codes (Indication of the maintainability of a system) Cumulative number of open issues (It measures whether the project is “converging”) Error density, that is, the number of errors found per source line of code. (It helps understand whether the development process has some systematic faults.) spm - ©2014 adolfo villafiorita - introduction to software project management 28 Managing Changes, Risks, and Quality Goals of the Unit Requests for changes and changes will occur in your project The goal of this unit is understanding: – The importance of keeping a project under scope – How request for changes can positively or negatively influence your project – The techniques to manage changes spm - ©2014 adolfo villafiorita - introduction to software project management 30 The Framework The scope document formalizes the goals of a project Ideally, once the goals are fixed, the project should move on to the design/implementation phase and achieve the project goals, through a progressive refinement Any deviation from such course of action is a perturbation (it changes goals, plans, costs, outputs, work to be performed, …) Changes, however, are inevitable The goal of a sound project management, therefore, is ensuring that the change process is properly managed spm - ©2014 adolfo villafiorita - introduction to software project management 31 Fundamental Concepts Change Control is the set of practices to ensure request for changes are properly taken care of Configuration Management is the set of practices to ensure project outputs remain coherent over time Change Control and Configuration Management span over the lifecycle of the project outputs In software projects artifacts are extremely simple to change (e.g., editing a file) In software projects, connection with bug reporting/bug lifecycle spm - ©2014 adolfo villafiorita - introduction to software project management 32 Change Control Causes of Request for Changes Incompleteness or incoherencies in the project requirements or in the description of work A better comprehension of the system to be developed A technical opportunity A technical challenge A change in the external environment Non-compliance of a project deliverable spm - ©2014 adolfo villafiorita - introduction to software project management 35 A Change Control Process It runs in parallel to the other PM activities throughout the project spm - ©2014 adolfo villafiorita - introduction to software project management 36 Comments A change control board might be appointed to approve/reject changes The cost and risk of changes increase as the project moves to the delivery The process ensures a formal record is kept and and a clear procedure is set to evaluate the impact of changes Change and change management is embraced by agile methodologies (changes “treated” as requirements) spm - ©2014 adolfo villafiorita - introduction to software project management 37 Software Evolution Models What makes a Software System Software systems are made of many different artifacts (sources, libraries, external libraries, documentation, conversion scripts, databases) Software systems run in many different configurations (e.g., base/pro, versions 1 and 2, Linux/OSX/Windows) Two sources of complexity need to be addressed to develop or maintain a software product: – Identification of the artifacts – Evolution spm - ©2014 adolfo villafiorita - introduction to software project management 39 Linear Development Model Source Code replaces Source replaces Source Version 1 Code Code Version Version 2 3 produces produces produces Application Application Application Version 1 Version 2 Version 3 spm - ©2014 adolfo villafiorita - introduction to software project management 40 Branching Development Model Source Source Source Code Code Code Version 1.1 Version 2.1 Version 2.1 Source Source Source Code replaces Code replaces Code Version 1 Version 2 Version 3 produces produces produces produces produces Application Application Application Application Application Version 1 Version 1.1 Version 2 Version 2.1 Version 3 spm - ©2014 adolfo villafiorita - introduction to software project management 41 Software Development Models Linear development: – Only one version of an application is running at any given time (Example: one-offs; many web applications are one-offs) Branching development: – Various versions of an application are running at a given time spm - ©2014 adolfo villafiorita - introduction to software project management 42 Configuration Management Configuration Management Configuration Management (CM) is a set of activities running in parallel to the development process, whose goal is establishing and maintaining system’s coherency over time – Part of the project management plan – Helps define project standards and best practices spm - ©2014 adolfo villafiorita - introduction to software project management 44 Configuration Management Main Goals Being able to build a system from a consistent set of components Being able to retrieve a software component when needed (consider: storage time, storage means) Being able to view the history of changes a system has undergone Being able to retrieve a previous version of a system Remark: closely related to the change management process spm - ©2014 adolfo villafiorita - introduction to software project management 45 Some Examples A bug is reported by a user on a COTS software we have been selling for ten years. A client requests an enhancement to a one-off system we sold in 2005. We need to reproduce/understand an odd behavior of the control software of a space exploration probe which is now orbiting Jupiter spm - ©2014 adolfo villafiorita - introduction to software project management 46 Steps and Tools: Establish Baseline The first step is “establishing what a product is” A good CM requires to: – Clearly identify the items which constitute a product – Identify the relationships among these items – Choose an appropriate identification and numbering scheme for versions – Take “snapshots”: baseline records spm - ©2014 adolfo villafiorita - introduction to software project management 47 Steps and Tools: Manage Changes The second step is “maintaining coherency over time” A good CM process requires to: – Define the “baseline record” (the starting point) – Identify and approve requests for changes (see change control) – Formally record changes and history of each item – Maintaining old versions For family of products there could be different baselines. Changes might need to be applied to one or more baseline (consider a security fix to a browser) spm - ©2014 adolfo villafiorita - introduction to software project management 48 Steps and Tools: Considerations For software development, a version control system implements various of the functions described above Tools are not sufficient: an adequate process has to be in place Semantic versioning is an example of numbering schema spm - ©2014 adolfo villafiorita - introduction to software project management 49 Version Control Systems: Main Concepts Working version: the file (or set of files we are currently editing) Repository: the storage where all versions of a file (or set of files) are kept together with additional information Repository Repository' Files Files a.each do |x| puts x History Log a.each do |x| a.each do |x| puts x History Log endputs x Tags Tags end end #a.each do | x| #a.each do | x| #a.each do | x| #puts x #end #puts x #end Version N. Version N. #puts x #end a.each do | commit a.each do | commit x| puts x end checkout x| puts x end #a.each do | #a.each do | x| #puts x x| #puts x #end #end spm - ©2014 adolfo villafiorita - introduction to software project management 50 Version Control Systems: Main Concepts In the simple case (early VCS) each file would have an independent repository Coherence is kept by assigning the same tags to all artifacts constituting a baseline More recent VCS manage sets of artifacts in an integrated way Tagging is used to mark important baseline records A VCS typically support parallel access and editing of artifacts spm - ©2014 adolfo villafiorita - introduction to software project management 51 Risk Management Motivations When we looked at project selection we just took into account financial data In the scope management document we emphasized the importance of making our goals achievable, i.e. the A in SMART... however between achievable and achieved there is a big difference. In the planning phase we had to deal with various uncertainties (estimation) and tried to deal with them generically (e.g. time buffers) We stuck to one plan (the nominal plan), but the world is non-nominal: changes, both negative and positive, will occur! spm - ©2014 adolfo villafiorita - introduction to software project management 53 Risk Management Risk management collects techniques, know-how and processes to help identify, assess, manage, and monitor risks The objectives of Project Risk Management are to increase the probability and the impact of positive events and decrease the probability and impact of events adverse to the project. spm - ©2014 adolfo villafiorita - introduction to software project management 54 Risk Management: Some Goals Understanding whether a project is worth taking Help refining the budget for the project Increase chances of ending the project successfully Increase chances of terminating the project as planned: – Within scope – Within quality – Within budget – On time spm - ©2014 adolfo villafiorita - introduction to software project management 55 Risk Management: Two Definitions “Traditionally”: – Risk is the possibility of suffering loss In project management: – (Project) Risk is an event or condition that, if it occurs has positive or negative influence on an objective * Negative outcome: menace * Positive outcome: opportunity spm - ©2014 adolfo villafiorita - introduction to software project management 56 Risk Management Used in several fields, such as: – Finance – Insurance – Engineering (safety critical, security, …) Various standards recognize the importance of risk in software development: – ISO/IEC 12207 (Information Technology - Software life cycle processes) – UNI EN 29000-3 (Guidelines for the application of ISO 9001 to software development and maintenance) – UNI ISO 10006 (Guidelines for managing projects) Various techniques (FMEA, FTA, simulation, …) have been defined and adopted to assess it. spm - ©2014 adolfo villafiorita - introduction to software project management 57 Goals of the Unit Learning the techniques to identify, assess, prioritize, manage and control project risks Learning what are the most common risks in software development projects Learning how to budget for project risks spm - ©2014 adolfo villafiorita - introduction to software project management 58 Risk Management and Project Management The Risk Management Process Define Risk Define Management Identify Risks Classify Risks Management Standards Strategies Risk Management Risk Register Risk Register Risk Plan Starndars (updated) and Risk Matrix [ new risk identified ] Monitor Risks Risk Log (updated) It runs in parallel to the other PM activities throughout the project spm - ©2014 adolfo villafiorita - introduction to software project management 61 Defining Risk Management Standards Goal: describing how risk management will be structured and performed on the project. – Output: a document (or set of documents and templates) – Part of the project management plan – Helps define project standards and best practices spm - ©2014 adolfo villafiorita - introduction to software project management 62 Define Risk Management Standards The document includes, at a minimum: – The procedures to monitor and update risks – The procedures to apply contingency plans – Who is in charge of what Added value: – Definition of risk probabilities and impacts – Risk Categories or other sources to identify risks – Reporting formats A risk management plan could be standardized and adopted organization-wide Different projects require different levels of formality in risk management spm - ©2014 adolfo villafiorita - introduction to software project management 63 Risk Identification Goal: understanding what are the risk that could potentially influence the project and document their characteristics – Risk identification is an iterative process (new risks may be identified as the project progresses; old risks may become “obsolete”) – Output: Risk Register, basis for qualitative/quantitative risk analysis spm - ©2014 adolfo villafiorita - introduction to software project management 64 Risk Identification and Classification Process (iterative): – Collect: * identify specific project risks * describe the risk – Analyze: * Identify the root causes (do not misinterpret effects as causes) * Define the risk category (impact) and probability * Identify other useful characteristics: – When it can occur or frequency of occurrence – How it manifests Output: – Risk Register spm - ©2014 adolfo villafiorita - introduction to software project management 65 Risk Identification Techniques Meetings Document Analysis Risk Breakdown Structures, Checklists, Templates Analogy spm - ©2014 adolfo villafiorita - introduction to software project management 66 Boehm’s Top Ten Causes for Project Failures Boehm developed a list of the ten most common causes for projects to fail Some of the causes mentioned in the list can be used as starting point to identify the risks applicable to a project at hand Risks include: – Personnel or subcontractors Shortfall – Unrealistic schedule and budget – Developing the wrong software functions/user interface – Gold plating (getting priorities wrong) – Ineffective change control – Technical risks spm - ©2014 adolfo villafiorita - introduction to software project management 67 Root Cause Analysis Techniques Cause-Effect Diagram (Ishikawa) Fault Trees/Failure Modes and Effect Analysis Machine Method Material Risk Measurement Man Mother Nature spm - ©2014 adolfo villafiorita - introduction to software project management 68 Fishbone Diagrams: Some starting points The 6 M's: – Machine, Method, Materials, Measurement, Man and Mother Nature (Environment) (recommended for manufacturing industry). The 8 P's: – Price, Promotion, People, Processes, Place / Plant, Policies, Procedures & Product (or Service) (recommended for administration and service industry). The 4 S's: – Surroundings, Suppliers, Systems, Skills (recommended for service industry). spm - ©2014 adolfo villafiorita - introduction to software project management 69 Risk Assessment and Risk Management Strategies Risk Assessment Goal: prioritize risks according to their impact and likeness on the project – Output: a prioritized list of risks (priority defined according to probability and impact) – Information on whether a project is worth taking – Information about what risks must be monitored spm - ©2014 adolfo villafiorita - introduction to software project management 71 Probability/Impact Frequency Why projects don’t have risks here? Why projects do not have risks Impact here? spm - ©2014 adolfo villafiorita - introduction to software project management 72 Techniques Qualitative risk analysis – Simpler – Can be used when no precise information about probabilities of risk is available Quantitative risk analysis – More systematic – Suitable for mathematical analysis – Provide figures on the (economical) impact of risks spm - ©2014 adolfo villafiorita - introduction to software project management 73 Qualitative Risk Analysis Qualitative Risk Analysis Start from – Risks Management Standard which define the scales to be adopted for probability and impact – The outputs of the risk identification phase (during which we assigned a probability to each risk) Highlight most significant risks: – By organizing risks into a risk matrix – By scoring risks Output: – Assess whether the project is worthwhile. – Decide what risks must be monitored spm - ©2014 adolfo villafiorita - introduction to software project management 75 Risk Matrix Negligible Low Moderate Severe Catastrophic Very High R1 R5 High R2 R6, R7, R8 Moderate R3 Low R4 Very Low R9, R10 spm - ©2014 adolfo villafiorita - introduction to software project management 76 Risk Matrix Decide where to set the “bar”... Negligible Low Moderate Severe Catastrophic Very High R1 R5 High R2 R6, R7, R8 Moderate R3 Low R4 Very Low R9, R10 spm - ©2014 adolfo villafiorita - introduction to software project management 77 Risk Matrix RED: Require special treatment (or drop the project) ORANGE: Need close monitoring GREEN: LOW: Standard in a project... nuisances Negligible Low Moderate Severe Catastrophic Very High R1 R5 High R2 R6, R7, R8 Moderate R3 Low R4 Very Low R9, R10 spm - ©2014 adolfo villafiorita - introduction to software project management 78 Risk Scoring Define classes of probabilities and classes of qualitative or numeric classes of impact Example – Probability: very low, low, moderate, high, very high – Impact: negligible, low, moderate, severe, catastrophic – Risk Score: low, medium, high (see previous slide) or numeric: SCORE = P xI Very Low 0.1 1 Negligible 0.1 1 Low 0.3 2 Low 0.3 2 Moderate 0.5 3 Moderate 0.5 3 High 0.7 4 Severe 0.7 4 Very High 0.9 5 Catastrophic 0.9 5 spm - ©2014 adolfo villafiorita - introduction to software project management 79 Socially constructed risk Two problems with qualitative risk – Modelers: we are “risk illiterate” (we believe some things are riskier than others, sometimes even when statistics tell us otherwise) – Models: who says what the probabilities are? How do we calculate the risk exposures objectively? (projects are one-offs) spm - ©2014 adolfo villafiorita - introduction to software project management 80 Examples of risks: Causes of Death Heart disease: 597,689 Nephritis, nephrotic syndrome, Cancer: 574,743 and nephrosis: 50,476 Chronic lower respiratory Influenza and Pneumonia: diseases: 138,080 50,097 Stroke (cerebrovascular Intentional self-harm (suicide): diseases): 129,476 38,364 Accidents (unintentional injuries): 120,859 Alzheimer's disease: 83,494 Diabetes: 69,071 Source: http://www.cdc.gov/nchs/fastats/lcod.htm (2011 data) spm - ©2014 adolfo villafiorita - introduction to software project management 81 Risk Management Strategies a.k.a. Risk Response Planning: how do we take care and exploit risks Risk Management Strategy Goal: find a treatment for the unacceptable risks and decide the strategies to apply for the remaining risks, should they occur during the project – Output: a plan with only acceptable risks – A contingency plan for each remaining significant risk spm - ©2014 adolfo villafiorita - introduction to software project management 83 The Scenario RED: Require special treatment (or drop the project) ORANGE: Need close monitoring GREEN: LOW: Standard in a project... nuisances Negligible Low Moderate Severe Catastrophic Very High R1 R5 High R2 R6, R7, R8 Moderate R3 Low R4 Very Low R9, R10 spm - ©2014 adolfo villafiorita - introduction to software project management 84 Strategies: Menaces Avoid – Change the plan to eliminate the threat (increase time, relax objectives, take corrective actions - increase time to do requirements) Transfer – Shift the negative outcome to a third party. It transfers responsibility, it does not eliminate the risk (insurance, contracts to transfer liability… they require to pay you a price) Mitigate – Reduce probability or impact (often better than trying and repair the damage; prototyping) spm - ©2014 adolfo villafiorita - introduction to software project management 85 Strategies: Opportunities Exploit – Eliminate uncertainty relate to the occurrence of the opportunity (e.g. assign more talented people, provide better quality) Share – Allocate responsibility of exploitation to a third party (joint-ventures, partnerships, …) Enhance – Modify the size of an opportunity by increasing probability and/or positive impact spm - ©2014 adolfo villafiorita - introduction to software project management 86 Strategy: common Accept – Passive: just let the team deal with the risks – Active: provide some buffer (time, money, …) Why?... Low impact or probability... Simpler to deal with the risk, if it occurs than planning a response in advance spm - ©2014 adolfo villafiorita - introduction to software project management 87 Risk Response Planning: Outputs Risk Response Plan: – Strategy (strategies) for dealing with the risks: must be concrete! – Triggers (elements used to monitor and understand whether a risk has occurred) – People responsible of monitoring the risk – People responsible of applying contingency plans spm - ©2014 adolfo villafiorita - introduction to software project management 88 The Risk Register The most common tool to list and manage risks is a spreadsheet One row per risk Each risk characterized by: – ID, Title, Description – Risk Category (if you are inclined to classifications) – Probability, Impact and, possibly, Score (PxI) – Root cause – Time-frame – Monitoring modalities (periodicity, person, reporting) – Status (active, occurred, inactive) spm - ©2014 adolfo villafiorita - introduction to software project management 89 Risk Monitoring and Control a.k.a. Risk Response Planning: how do we take care and exploit risks Risk Monitoring and Control Input: – The risk register Process – Analyze deviations from the nominal plan – Identify causes – Evaluate corrective actions – Modify current plan Mind: – Planned risks must be dealt with as above (use contingency plans) – Unplanned risks require the full process! spm - ©2014 adolfo villafiorita - introduction to software project management 40 Conclusions and The main risks of... Risk Management! Some Common Errors During the Planning Phase: – Not identifying a maximum risk value * Give up a project if too risky – Not writing a balanced risk management plan * Size and complexity have to be at the right level for the project – Misinterpreting e ects as causes * You end up caring for the wrong event and not looking at the actual problem * Example 1: We might be late with the project * Example 2: We may be charged 100.000 euros as a penalty spm - ©2014 adolfo villafiorita - introduction to software project management 93 ff Some Common Errors During Risk Monitoring: – Risk homeostasis: we tend to increase our risk-taking – Anchors and frames: we tend to stick to anchor and frames overlooking opportunities or the need to change course. An example is sticking to past decisions (as an anchoring mechanism) – Sunk costs: an incorrect economical assumption (“I have spent so much… it is more convenient to keep going!”) – Cognitive dissonance: we do not like inconsistencies; our brain creates consistent theories, sometimes altering (or not considering) all facts. (“I know smoking is bad … but (another) cig won’t hurt me”) spm - ©2014 adolfo villafiorita - introduction to software project management 94 Some Common Errors During Risk Monitoring: – Do not apply contingency plans * Dealing with risk when they occur is more error-prone than think about the strategies before they occur – Do not involve actors * Make sure stakeholders understand consequences of the risk (share the risk) * involve stakeholders in dealing with them – Do not update the plan * Helps keeping the contingency plans really applicable spm - ©2014 adolfo villafiorita - introduction to software project management 95