Project Execution, Monitoring, and Control PDF

Project Execution, Monitoring, and Control Goals of the Unit Plans have little value if not executed and, during execution, monitored and updated to reflect the current situation This unit introduces: – The activities to put a plan into practice – The techniq...

Project Execution, Monitoring, and Control Goals of the Unit Plans have little value if not executed and, during execution, monitored and updated to reflect the current situation This unit introduces: – The activities to put a plan into practice – The techniques for monitoring and controlling your plans (progress and costs against schedule and budget) – Earned Value Analysis, a technique which allows the project manager to monitor a project in an integrated way spm - ©2014 adolfo villafiorita - introduction to software project management 2  Initiate Plan Execute & Close Monitor Assess Formalize Collect Close Feasibility Goals Outputs Monitor Goals, Cost and Develop Release Schedule Define Kick Off Schedule Activities Define Costs [Obtain Approval] Change Control & Configuration Management Quality Management Risk Management Human Resource Management Project Execution Project Execution Project execution is where work is performed There are three main management activities: – Kicking activities off – Collecting the output of activities – Collecting information about the project health spm - ©2014 adolfo villafiorita - introduction to software project management 5 Kicking Activities Off Goal: – Ensure there is a formal start for a significant portion of a project – Ensure the team is aligned on the goals and modalities of the activities being started The main mean is a kick-off meeting In general, any communication mean can be used (but it risks being less effective than a kick-off meeting) Choose an adequate level of granularity spm - ©2014 adolfo villafiorita - introduction to software project management 6 Collecting the Output of Activities Goal: – Systematic collection of project outputs (deliverables) – Occasion to assess the lesson learned For software projects the main mean to collect project outputs is a repository + tagging/versioning A meeting to assess the lesson learned can also be used to “formalize” the collection of outputs spm - ©2014 adolfo villafiorita - introduction to software project management 7 Collecting Information about the Project Status Goal: – Systematic collection of data to assess the project status It can be performed on a regular basis (in which case the frequency has to be chosen according tot he project size) It can be performed on a need basis (for exceptional events, e.g., risks) Quantitative data can be collected based on the monitoring means Qualitative data (e.g., team morale, “feeling” about the status or difficulty of a given task) must also be collected spm - ©2014 adolfo villafiorita - introduction to software project management 8 Project Monitoring and Control Introduction Goals: – For the project: assessing project status (scope, time, cost, quality,...), analyzing deviations, and taking corrective actions, if necessary – For the organization: collecting data helps building a better and more accurate plans for future projects Process (on a regular basis): – Collect. Get the data about the current status of your project. – Measure and Compare. Compare with baseline plan, highlight any deviation, make a projection based on current data. – Assess and Re-plan. Decide whether corrective actions are necessary. If so, plan, document, and take the corrective actions. spm - ©2014 adolfo villafiorita - introduction to software project management 10  Monitoring and controlling cycle deviations & Plan P1 compare replan P2 assessment (baseline) (new baseline) … and the cycle repeats... Monitoring A1 (actual plan) describes captured by Actual World work Approaches Focus: – Here we focus on schedule, costs, and progress Non-integrated approach: – Monitor schedule: understand whether we are late or early – Monitor costs: understand whether we over or under budget – Simple, but partial views Integrated approach: – Earned Value Analysis: measure schedule, costs, and progress together – More complex, but a more comprehensive view spm - ©2014 adolfo villafiorita - introduction to software project management 12 Monitoring Schedule Basic Concepts Baseline (planned values): – A snapshot of the plan at a given time (plan at t1, plan at t2, …) – Many baselines can be taken Actual Values – Actual status of the schedule – Actual start, actual end, actual effort/actual progress planned start planned end actual start actual end spm - ©2014 adolfo villafiorita - introduction to software project management 14 The Process 1. Build the plan 2. Save a baseline 3. On a regular basis, assess the plan: 1. Actual start and end of an activity 2. Actual effort spent on the activity 3. Technical progress (may be difficult to assess) 4. Re-plan: 1. Estimate effort and duration to end 2. Technique 1: efficiency with which actual effort has been expressed w.r.t. planned effort 3. Technique 2: efficiency with which technical progress is expressed w.r.t. planned progress 4. Share the new plan, and GOTO 2 spm - ©2014 adolfo villafiorita - introduction to software project management 15 Collecting Effort Data Depending on the level of formality... people may be required to provide data about effort spent on activities Usually best on a weekly basis Need to reference activities of the plan It will contain “noise” John Doe W1 W2 W3 W4 Requirements M1 30 6 Requirements M2 30 6 Meeting 2 2 6 2 Research 4 4 4 4 Indirect activities 2 spm - ©2014 adolfo villafiorita - introduction to software project management 16 Monitoring Costs Cost control, the simple approach The budget table defines your baseline Actual costs define your current status It can be split over years (or reporting periods) CBS Item Budgeted Actual Status New Budget Hardware €10,000.00 €5,000.00 €5,000.00 €5,000.00 Software €4,000.00 €2,000.00 €2,000.00 €2,000.00 Travel €5,000.00 €6,000.00 -€1,000.00 €1,000.00 Project Bfr €3,000.00 €3,000.00 €1,000.00 Total €22,000.00 €13,000.00 €9,000.00 €9,000.00 Overruns drawn from other funds (e.g. project buffer, a different CES item) or from other projects spm - ©2014 adolfo villafiorita - introduction to software project management 18 Remarks Advantages: – Relatively simple (however, delays between commitment of expenditures and cash flow) – For various CES items probably the best way of monitoring (e.g. hardware, software,...) Disadvantages: – Not sufficient to have an idea on the overall status of the project (will we make it with the remaining money?) spm - ©2014 adolfo villafiorita - introduction to software project management 19 Earned Value Analysis Earned Value Analysis Earned Value Analysis provides an integrated view of the project by measuring planned effort (costs), actual progress (earned value), and effort (actual costs) in terms of monetary values Measuring plan, work, and progress with the same unit makes them comparable Useful because: – Progress becomes comparable with effort – Budget and actual costs are put in context (being under budget is not necessarily good, if the technical progress is even lower) spm - ©2014 adolfo villafiorita - introduction to software project management 21 Assumptions and Definitions Assumptions: – Manpower = Cost: plotting effort or cost is equivalent – Corollary: Actual manpower = Actual Cost – Progress = Money Definitions: – Planned Value: the cumulative costs planned for the project. Also called: Budgeted Costs of Work Scheduled – Actual Costs: the cumulative costs actually incurred into. Also called: Actual Costs of Work Performed – Earned Value: the actual progress, expressed as the quantity of planned value which has generated results spm - ©2014 adolfo villafiorita - introduction to software project management 22 Planned Cost Computation Planned Name Time Cost A 1000 A B 2000 B C 400 C A 1000 500 500 B 2000 1000 1000 C 400 100 100 100 100 Total 500 1500 1100 100 100 100 Cumulative 500 2000 3100 3200 3300 3400 4000 3400 3100 3200 3300 3000 2000 2000 1000 500 0 M1 M2 M3 M4 M5 M6 spm - ©2014 adolfo villafiorita - introduction to software project management 23 Earned Value Computation Rule 1. – Earned value should be determined by examining products Rule 2. – 50/50 Rule (50% of Planned Value at start and 50% at end) – 20/80 Rule (20% at start and 80% at end) – 0/100 Rule (0% at start and 100% at end) spm - ©2014 adolfo villafiorita - introduction to software project management Earned Value Analysis Money BAC = project budget AC (Actual Costs) PV (Planned Value) Cost Variance Schedule Variance EV (Earned Value) Time monitoring planned end date spm - ©2014 adolfo villafiorita - introduction to software project management 25 Some Interesting Points and Metrics BAC = Budget at Completion SV: Schedule Variance (BCWP-BCWS) – A comparison of amount of work performed during a given period of time to what was scheduled to be performed. – A negative variance means the project is behind schedule CV: Cost Variance (BCWP-ACWP) – A comparison of the budgeted cost of work performed with actual cost. – A negative variance means the project is over budget. spm - ©2014 adolfo villafiorita - introduction to software project management 26 Cost Performance Index (CPI) EVt CPIt = \mbox{\em CPI}_t = \frac{\mbox{\em EV}_t}{\mbox{\em AC}_t} ACt CPI (Cost Performance Index) compares work performed to actual costs How much are we getting for each euro we spend? CPI > 1 Project is efficient CPI < 1 Project is inefficient spm - ©2014 adolfo villafiorita - introduction to software project management Schedule Performance Index (SPI) EVt SPIt = PVt SPI (Schedule Performance Index) compares work performed to work planned How fast does the project progress w.r.t. how fast we expected it to be? SPI > 1 Project is ahead of schedule SPI < 1 Project is behind schedule spm - ©2014 adolfo villafiorita - introduction to software project management Cost Schedule Index (CSI) CSIt = CPIt SPIt EVt EVt CSIt = ⇤ ACt P Vt CSI: Cost Schedule Index (CSI=CPI x SPI) The further CSI is from 1.0, the less likely project recovery becomes spm - ©2014 adolfo villafiorita - introduction to software project management Measuring SPI and CPI SPI early, but early and early over budget under budget 1 exactly like scheduled and budgeted late and late, but late over budget under budget CPI over budget under budget 1 spm - ©2014 adolfo villafiorita - introduction to software project management 30 Earned Value Analysis: Example Example Remarks: – Lower part = baseline; upper part: actual & progress Questions: – Are we late? – Are we over budget? spm - ©2014 adolfo villafiorita - introduction to software project management 32 Example Activity 1: as scheduled (time) Activity 2: started late; ahead of schedule Activity 3: started earlier; progress same as time elapsed Activity 4: not started yet spm - ©2014 adolfo villafiorita - introduction to software project management 33 Example: BCWS (Planned Value) Paint Wall 800 800 800 800 Paint Ceiling 800 800 800 800 Refurnish 400 400 Clean 400 Budgeted Cost of Work 800 1600 2400 4000 4800 5600 6800 7200 7600 Scheduled (BCWS) € 8,000 € 6,000 BCWS € 4,000 € 2,000 €0 w1 w3 w5 w7 Untitled 1 spm - ©2014 adolfo villafiorita - introduction to software project management 34 Example: BCWP (Earned Value) Paint Wall 1600 1600 Paint Ceiling 0 1600 Refurnish 400 Clean Budgeted Cost of Work 1600 1600 1600 3200 5200 Scheduled (BCWS) € 8,000 € 6,000 BCWS € 4,000 BCWP € 2,000 €0 w1 w3 w5 w7 Untitled 1 spm - ©2014 adolfo villafiorita - introduction to software project management 35 Comments Ahead of schedule on week 1 because of the noise of the 50%-50% rule (analogously the delay on w2 and w3) w4: we are behind schedule (activity 2 did not start as expected) w5: we are again ahead of schedule, because of activity 3. Since the 50%-50% rule only counts start and end of activities, the fact that progress in activity 2 is better than expected is not taken into account in the EVA graph spm - ©2014 adolfo villafiorita - introduction to software project management 36 Example: ACWP (Actual Costs) Paint Wall 900 900 900 900 Paint Ceiling 0 2000 Refurnish 400 Clean Actual Cost of Work 900 1800 2700 3600 6000 Performed (ACWP) € 8,000 € 6,000 BCWS € 4,000 BCWP ACWP € 2,000 €0 w1 w2 w3 w4 w5 w6 w7 w8 spm - ©2014 adolfo villafiorita - introduction to software project management 37 Recap Paint Wall €800.00 €800.00 €800.00 €800.00 Paint Ceiling €800.00 €800.00 Refurnish Clean Budgeted Cost of Work Scheduled (BCWS) €800 €1,600 €2,400 €4,000 €4,800 Budgeted Cost of Work Performed (BCWP) €1,600 €1,600 €1,600 €3,200 €5,200 Actual Cost of Work Performed (ACWS) €900 €1,800 €2,700 €3,600 €6,000 CPI (BCWP/ACWP) 178% 89% 59% 89% 87% SPI (BCWP/BCWS) 200% 100% 67% 80% 108% Cost Variance €700 -€200 -€1,100 -€400 -€800 Schedule Variance €800 €0 -€800 -€800 €400 spm - ©2014 adolfo villafiorita - introduction to software project management 38 Comments Various noise due to the 50%-50% rule (e.g. w1) Data shows that we are now a bit over budget, but early in schedule (last column). However: – Actual costs efficiency is due to the 50%-50% rule on activity 2 (we accrued 1600)... the data will get more accurate when we finish activity 2 (expenditure will likely be 4000 euros and BCWS 3200) spm - ©2014 adolfo villafiorita - introduction to software project management 39 Agile Monitoring and Control Burndown Chart Agile Estimation Practices Simplifying a bit, SCRUM and other agile methodologies: – Move away from the effort as a perfect measure – Structure development in sprints of fixed duration – Focus on remaining work, rather than work to be performed Examples: – Extreme Programming estimates using “ideal programming hours” – SCRUM estimates using “points” (an abstract measure, which deliberately moves away from traditional effort estimations) spm - ©2014 adolfo villafiorita - introduction to software project management !3 Agile Estimation Practices The estimation process assigns a number of ideal hours or points to each feature to be implemented Features are then assigned to a sprint, determining the number of points to be burned During the sprint points are “burned” as features are delivered. One management goal is measuring the speed at which features are delivered and ensuring the estimation process keeps the speed constant between sprints spm - ©2014 adolfo villafiorita - introduction to software project management !4 Agile Progress Monitoring Burndown 160 140 120 100 ideal burndown 80 (constant speed) actual burndown 60 (actual progress) 40 20 0 ESTIM. 1 2 3 4 5 6 7 8 9 spm - ©2014 adolfo villafiorita - introduction to software project management !5 Agile Earned Value Analysis Agile EVM A mix of Agile and Earned Value Management Motivations (*) – Substantial: * Are we making enough progress, but only by blowing the budget with overtime? * If we are running late, was it due to problems in the project, or was it due to other projects “borrowing” our people? – Substantial/Formal: * Certain projects require the application of EVM (e.g. defense projects in the US > 20M$) At least two different methodologies have been proposed (we will look at one) (*) Source: http://www.agilekiwi.com/EarnedValueForAgileProjects.pdf Agile Earned Value: an Approach The following method proposed by: John Rusk - Earned Value for Agile Development http://www.agilekiwi.com/ EarnedValueForAgileProjects.pdf It works with SCRUM Simple and informal: – No special terms; it just assigns colors to the different curves (simpler) – It uses percentages rather than money Three lines: – The grey line (= planned value) shows the progress that we expect to make, as a percentage of the total project. It starts at zero and slopes up to 100% at the end of the project. – The green line (= earned value) shows how much of the product we have built, as a percentage of the total product size. – The red line (= actual costs) shows the cost we have incurred so far, as a percentage of the total project budget.  8 80% 9 90% 10 100% 11 100% 12 100% 13 100% Example 120% Planned Value Earned 100% Value Actual Costs 80% 60% 40% 20% 0% 1 2 3 4 5 6 7 8 9 10 11 12 13 Understanding the Plot In a perfect world, the three lines are perfectly aligned The green line measures the schedule: – If green above the grey line: ahead of schedule – If green below the grey line: behind schedule The red line measures efficiency: – If red above the green line: spending the budget faster than we’re building the software. Cost overrun! – If red below the green line: building the software faster than we are spending the money. Under budget! Computing the Lines Planned Value is computed from the backlog (sum of story points). Velocity is assumed constant and the result is a straight line (simplification, but good enough) Actual Costs are computed from costs of hours worked as a percentage of the total project budget. Earned value is the sum of completed story points. No points for partial completion of a task. This practice is consistent with that normally used on agile burn charts and is referred to as the 0/100 rule in EVM. Only working software features earn value: you only score points when it is designed, built and tested. Some Remarks The chart works best when it covers a period of time that is big enough to feel like the “big picture” (e.g., 3 to 6 months) Different levels of granularity are possible (to show daily progress, for instance) The technique requires scope to be available up-front (the backlog has to be known before starting the project) Delivery of value must be linear (e.g. testing is not at the very end)... otherwise the 0-100% rule does not allow one to measure progress. Managing Changes Two types of changes: – Nuances in requirements * Approach: do not account them (they are “sunk” in the known requirements) – Actual scope changes (e.g.: new ideas) * Approach: Measure scope changes and revise EVM plot 100% 100% new scope old scope scope increases by 20% time time new project end Project Closing Goals of the Unit All projects come to an end Many project, however, live a long period in “limbo” land, not active but neither properly closed The goal of a good project manager is ensuring projects are properly closed Goals of the Unit: – Understanding what activities need to be performed to close a project – Understanding why projects are not properly closed – Understanding the risks of not properly closing a project spm - ©2013 adolfo villafiorita !2  Initiate Plan Execute & Close Monitor Assess Formalize Collect Close Feasibility Goals Outputs Monitor Goals, Cost and Develop Release Schedule Define Kick Off Schedule Activities Define Costs [Obtain Approval] Change Control & Configuration Management Quality Management Risk Management Human Resource Management Types of Project Closing Termination by integration and termination by addition Successful cases: project outputs integrated and/or used as input for another project/production Termination by starvation Project ends because resources run out. Termination by extinction Termination by management because the project failed (objectives not met, superseded, not profitable) spm - ©2013 adolfo villafiorita !4 Project Closing Project closing is the last phase of a project, when the project outputs are handed over to the stakeholders, contractual agreements properly taken care of, and project records elicited and stored for future reference Main goals: – Ensuring project outputs can be used – Ensuring there are no pending/further obligations – Taking stock and learning spm - ©2013 adolfo villafiorita !5 Why Projects are not Properly Closed For unsuccessful projects – Little interest by the team For all projects – Decreasing interest by the project team – Cost of performing closing activities – Closing activities require little or no creativity – Underestimation of how much implicit knowledge there is – Underestimation of how fast know-how can get lost – Reluctance to release resources for opportunistic reasons – Emotional factor spm - ©2013 adolfo villafiorita !6 Project Closing Process Getting client acceptance Installing Project Deliverables Archiving old Deliverables Documenting the Project Performing a Financial Closure Performing Post-Implementation Audit Releasing Staff spm - ©2013 adolfo villafiorita !7 Getting Client Acceptance Ceremonial acceptance – No formal procedure or formal record for accepting project deliverables – Scenarios: gentlemen agreement; reaching project deadlines Formal acceptance – Formal procedure for accepting project deliverables – System testing/client approval spm - ©2013 adolfo villafiorita !8 Post-Implementation Audit (Post-mortem) We hate doing the same mistakes over and over again The goal of a post-mortem is a critical analysis of the project in order to learn and improve, avoiding to repeat the same mistakes Different formats and levels of formality are possible Unsuccessful projects provide a lot of information Useful lessons also from successful projects (what worked, what we could have done better) spm - ©2013 adolfo villafiorita !9 Structure of a Post-Mortem Conduct project survey Elicit main issues and strengths of the project Collect objective information Elicit quantitative measures about the project Hold a debriefing meeting Conduct a project history day Find the root causes of problems Publish the results Make sure your organization, your team, and you can learn from the experience spm - ©2013 adolfo villafiorita !10 Post-Mortem Metrics A quantitative assessment allows a more precise evaluation of the project Data can be used for future estimations Metrics include: Sheet1 Cost Metrics Schedule Metrics Quality Metrics Planned Efort and Original Schedule Estimated SLOC Actual Efort and Actual Final Schedule SLOC History of changes to History of schedule Errors at each stage requirements and code slippage events spm - ©2013 adolfo villafiorita !11 Post-Mortem Results Structure The outputs of a post-mortem audit are published in a document The document can be used to disseminate the lessson learned and to work as a reference for future similar project A post-mortem report can be organized as follows: – Project description: information about the project, to give context – The good: what worked well – The bad: the three worst factors that impeded the teams to meet goals – The ugly: a prescription for improvement spm - ©2013 adolfo villafiorita !12 Releasing Staff Transition to new activities can be disruptive for the team (consider, e.g, a project lasting for years) Two important aspects: – Ensuring proper recognition to experience gained in the project and results obtained – Ensuring proper tasks are assigned to the team members spm - ©2013 adolfo villafiorita !13