2024 CE6304 Risk & Uncertainty PDF

CENG6304 Planning & Economic Evaluation of Transport Projects Week 9 Fitsum Teklu Last few weeks (Social) Cost Benefit Analysis Alternative Methods HDM 2 Treatment of Risks & Uncertainty Week 9 3 There are several sou...

CENG6304 Planning & Economic Evaluation of Transport Projects Week 9 Fitsum Teklu Last few weeks (Social) Cost Benefit Analysis Alternative Methods HDM 2 Treatment of Risks & Uncertainty Week 9 3 There are several sources of risk and uncertainty in appraisal Objectives Traffic risk Current Conditions External factors Assess Problems (using indicators) Future Conditions Economic growth Cost escalation (e.g. foreign currency & fuel cost) Possible Strategies Possible Measures Political stability Covid Predict Impacts Network effects Compare Solutions Other changes to the transport system which impact the impact of project/policy Implement etc. Assess Performance Monitor / Evaluate 4 Traffic Risk A key step in a systematic approach to designing transport policy/infrastructure interventions involves estimating the (traffic) demand impact of alternative options consider an option in which a new toll road is designed, built, financed and operated (for 30 years) by a private sector investor Some sources of uncertainty Uncertainty of inputs such as the value of travel time savings, GDP forecasts, vehicle operating costs (considering improvements in EV/battery technology) for the next 30 years Errors in model specification and parameters Did model include alternative un-tolled route options? What is the elasticity of demand to (say, inflation-indexed) price increases? And how was it derived? Read: Robert Bain’s interview from 2013 on toll road forecasts 5 External Risks Impact of fluctuations in fuel prices, economic conditions, currency exchange rates and impact of pandemics (e.g. Covid) on transport projects/demand Economic factors are key drivers of demand Economic downturns such as that of 2008/9 financial crisis were one of the main reasons that led to shortfalls in toll road revenue that ultimately led to the toll road companies filing for bankruptcy Exchange rate risks are key particularly in terms of servicing debts secured from international lenders The Covid pandemic of 2020 has led to significant reduction in public transport usage 6 Other risks Technical risk construction delays, equipment failure, design flaws and their impact on project timelines and costs. Operational Risk What else can you think of? Maintenance and operational challenges. And how would the risk affect e.g. bridge maintenance costs affected by harsh weather conditions. the economic benefits or costs of the project? Technological Uncertainty Rapid advancements in transport technologies. e.g. uncertainty around electric vehicle adoption, affecting revenue generated from fuel taxes. Policy and Regulatory Uncertainty Changing laws, regulations, and funding priorities, such as current policy of discouraging the import of diesel/petrol vehicles into Ethiopia. Social and Political Risks public opposition, community disruption, social inequalities, policy changes, government instability, regulatory uncertainty 7 Techniques for understanding of/accounting for uncertainty Judgement-based approaches (Delphi) – see 3.26 to 3.30 of TAG Uncertainty Toolkit Scenarios Low, Medium and High Thematic scenarios: e.g. recently published Ethiopian Transport and Logistics Master Plan Sensitivity studies Assess how robust a decision is to changes in specific inputs (e.g. VoT, $ exchange rate) Monte Carlo Simulation-based risk modelling Optimism Bias Other decision-making approaches (Source: TAG Uncertainty Toolkit (2023) - https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/1139994/dft- uncertainty-toolkit.pdf) 8 Other strategies to manage risk Contingency Planning Risk transfer to private parties, via public-private partnerships (PPPs) E.g. private operators can assume traffic and revenue risk for toll roads. Insurance and Hedging Role of insurance and financial instruments (e.g. hedging against fuel price volatility) Stakeholder Engagement to identify risks Involving stakeholders (communities, experts, policymakers) in risk assessment could help identify issues that are unknown to the policy maker/consultant 9 Methods 10 Judgement based methods Use of expert judgment to assess project risks and uncertainties. Typically when analytical techniques are unfeasible due to time, resource or data constraints An example is the Delphi method which is a structured communication technique which is an interactive forecasting method that relies on a panel of experts. The experts answer questions in two or more rounds. After each round a facilitator provides anonymized summary of the experts forecast from the previous round along with the reasons provided for each judgments. Then, they are all encouraged to revise their earlier answers reflecting the views of the other experts. Disadvantage of this approach is that it could be highly subjective, and requires expertise to reasonably grasp the range of possible outcomes. 11 Scenarios A process of analysing future events by considering several alternative outcomes Each scenario outcome and pathway should be plausible, and scenario analysis itself observes the impact of different possible futures on a scheme’s strategic goals When there are several sources of uncertainty, the definition of the Core Scenario is important. The core scenario should Be based on published plans (not speculations) in terms of assumed other infrastructure and policies Reflect ‘firm and funded government commitments’ Reflect central projections of key exogenous demand drivers such as GDP Disadvantage includes the choice of scenarios being subjective and potentially not covering the full range of possible futures. It provides no information about the likelihood of each scenario occurring. Scenarios may not be evenly distributed around the most likely outcome, risking optimism/pessimism bias. 12 The 30-year Ethiopian Transport and Logistics Master Plan is based on three national economic scenarios Scenario 1 ”Go-Ahead” Smooth recovery from pandemic, etc. the “Homegrown Economic Reform Plan” of Ethiopia keeps its objectives, even though with some years of delay. Ethiopia is able to leverage on its points of strength and becomes a preferred destination of the Foreign Direct Investments, with significant injection of knowhow and new technologies. Scenario 2 ”Next Generation” Same macro as Scenario 1 technologies and the energy sources used in motorizing the transport means have shown in the last months an astonishing acceleration, thanks to the political actions in response of the global warming, and the major economies will invest unprecedented amounts of money in the re- conversion of their transport systems in order to reduce emissions Scenario 3 “Limits to Growth” CONTINGENCY Scenario if economic effect of the pandemic and of the war in Europe will be much worse than assumed in Scenarios 1 and 2. 13 Sensitivity Testing Assesses how robust a decision or a model output is to changes in specific inputs. This is particularly useful when there is a high level of uncertainty around key inputs and model parameters. Sensitivity testing is performed against variation in parameters which are judged to have a substantial effect on a model’s predictions, and/or be uncertain in their calibration. Disadvantage Standard practice is to run key sensitivities, but through testing can be resource intensive. Provides no information about the likelihood of different outcomes Only focuses on ONE DIMENSION of uncertainty in isolation, which risks obscuring additional sources of uncertainty 14 Monte-Carlo Simulation 1. Identify any uncertainty around key inputs; 2. Assign a range of potential input values (i.e. a probability distribution) to each input variable deemed to be uncertain: For simplicity and proportionality, a ‘triangular distribution’ is often used although there other may be appropriate (such as, uniform, normal, log normal, Poisson or negative exponential); A triangular distribution is widely used for risk quantification for continuous random variables. It has only one peak, and is entirely defined by three parameters: i. Lower percentile value – typically 10th percentile (i.e. 10% of observed values are below X); ii. Modal value (most likely) – represents the central case scenario that is most likely to occur; iii. Upper percentile value – typically 90th percentile (i.e. 10% of observed values are above Y); Triangular distributions may be skewed / asymmetric (i.e. not equally distributed about the mean). 3. Use software to run multiple iterations of these key variables (based on the assigned probability distribution) through the model in question, which produces a distribution of outputs. 15 Monte Carlo Simulation A digression 16 Generating random numbers following distributions Statistical distributions are represented by probability CDF density functions (PDFs) and cumulative density functions (CDFs) 1.0 The CDF of a random variable 𝑋 is a function 𝐹(𝑥) such that 𝐹(𝑥) = 𝑃(𝑋 ≤ 𝑥) Inverse Transform Sampling Method is the most common approach to generating random numbers from CDFs Generate a uniform random number 𝑈 between 0 and 1 Find the value of x such that F(x)=U, where F(x) is the CDF of the distribution The value of x is a random number sampled from the distribution 17 What is a mean BCR for the project assuming the different components follow a triangular distribution [a, c, b] cost/benefit items vary from a to b, with a mode at c (Billion Birr) Road Capital Cost [6, 6.7, 8] Vehicle Operating Cost Savings [1.8, 2, 2.25] Time Savings [4.5,6, 7] 18 Optimism Bias In the UK, several ex-post evaluations demonstrated systematic tendency for appraisers to be over-optimistic about key project parameters, including capital costs, operating costs, project duration and benefits delivery Optimising bias adjustments are made based on statistical modelling of similar past projects Read: https://assets.publishing.service.gov.uk/media/6093d448e90e0726f52fc54c/updating-the- evidence-behind-the-optimism-bias-uplifts-for-transport-appraisals.pdf 19 Example: Public Private Partnership for a tolled Expressway Identify sources of risks and uncertainty How would you take account of them in an appraisal / project planning setting? Who should take these risks? 20 Presentational issues 21 Analysts and scheme promoters should present uncertainty to decision makers Consistent and effective presentation is an integral part of uncertainty analysis and will help ensure such analysis feeds through to decision making 25% BCR Focus should be placed on what the uncertainty 20% analysis can tell decision makers, and support them 15% make more informed decisions 10% Assumptions used to create forecasts should be clearly 5% drawn out and explained 0% An uncertainty log should summarise all known 0.80 0.85 0.90 0.95 1.00 1.05 1.10 1.15 1.20 1.25 1.30 1.35 1.40 1.45 1.50 1.55 assumptions and uncertainties in the modelling and forecasting approach. If possible, this should be linked with the final output that is being used to make decisions [e.g. BCR, NPV, and IRR] 22 Error bars, Fan Charts and Violin Plots 23 Questions? [email protected] 24 Next week Monitoring & Evaluation 25 Thank you! 26