Topic 2 - Decision Making.pdf

Module M13AAE: Decision Making and Error in Aviation Topic 2: Decision Making 2.1 Introduction Decision making is a joint function of: • The features of the task, and • The users’ knowledge and experience relevant to those tasks Decision making can also be either supported or inhibited by the u...

Module M13AAE: Decision Making and Error in Aviation Topic 2: Decision Making 2.1 Introduction Decision making is a joint function of: • The features of the task, and • The users’ knowledge and experience relevant to those tasks Decision making can also be either supported or inhibited by the users’ systems, particularly the displays. It is a complex cognitive process (or processes) affected strongly by situational and environmental conditions (e.g. stress, task pace, criticality). For example, reflect on the differences in decision making approach you would adopt when deciding where to go on holiday compared with deciding what to do when you set fire to yourself when trying to impress your friends by flambéing steak on the BBQ…… There are two classical paradigms for the study of decision making, which we will examine in turn:   Classical Decision Making (CDM) Naturalistic Decision Making (NDM) We will then look briefly at the ways in which decision making can be improved and at the types of decision making errors. But before we start, you might like to take a look at this interview with Daniel Kahneman, a Nobel Prize winner on the subject of decision making: https://www.youtube.com/watch?v=mWaIE6u3wvw 2.2 Classical Decision Making The CDM approach was derived from the normative decision making models found in economics and statistics. Implicit in these models was the concept of an optimal decision. The decision maker is characterised as being completely rational, totally informed and infinitely sensitive. The decision maker’s actions are characterised as follows • • • They first formally recognises and describes what is known (and knowable) about the problem space Then they formally analyse options by applying principles of utility, uncertainty and risk Finally they choose the optimal solution from the range of alternatives derived The alternatives under consideration include a set of potential actions; a set of events; a set of consequences for each combination of action and event, and a set of probabilities for each combination of action and event. Decisions outcomes are then represented and rationally evaluated in terms of these elements resulting in the choice of a logical course of action. Coventry University: Academic Year 2015/2016 © Coventry University 2015 Module M13AAE: Decision Making and Error in Aviation 2.3 Subjective Expected Utility SEU theory SEU is an attempt to model how people make decisions in the presence of risk. It has two subjective components to it:   A subjective probability estimate relating to the likelihood of success (or failure) A utility function: o If a £10 note is found by a beggar has much higher utility (value) than if the same £10 note is found by Bill Gates The expected utility of an outcome is derived from its utility weighted by its probability of Occurrence:   A risk neutral decision maker will regard the expected utility of a bet coming to fruition to be equal to the utility of the stake’s value A risky decision maker the utility of the expected value of the winning bet will be greater than the utility of the value of the stake - Vice versa for the risk averse decision maker The SEU approach would suggest that a decision maker analyses all the various options and picks the one that has greatest expected utility. 2.4 Janis and Mann – The skilled decision maker Janis and Mann proposed a seven-stage approach to decision making. The decision-maker, to the best of his ability and within his information processing capabilities: • • • • • • • Thoroughly canvasses a wide range of alternative courses of action Surveys the full range of objectives to be fulfilled and the values implicated by the choice Carefully weighs whatever he knows about the costs and risks of negative consequences, as well as the positive consequences, that could flow from each alternative Intensively searches for new information relevant to further evaluation of the alternatives Correctly assimilates and takes account of any new information or expert judgement to which he is exposed, even when the information or judgement does not support the course of action he initially prefers Re-examines the positive and negative consequences of all known alternatives, including those originally regarded as unacceptable, before making a final choice; and Makes detailed provisions for implementing or executing the chosen course of action, with special attention to contingency plans that might be required if various known risks were to materialise Whilst you may recognise this as the type of approach that you think you adopt when making an important decision such as buying a new car or choosing a new house (after all you, like me, are an expert decision maker of course!), there are some problems with this approach. It assumes: • The decision maker has access to complete and reliable information concerning all aspects of the problem Coventry University: Academic Year 2015/2016 © Coventry University 2015 Module M13AAE: Decision Making and Error in Aviation They can process all of the data to define the problem so that it is ultimately clear and unambiguous They can identify all possible solutions to the problem and be aware of their consequences The decision maker is capable of evaluating the outcomes by applying some kind of weighting to these options • • • It is also implied that the decision maker is neither time constrained in undertaking this process nor under any degree of stress. There often isn’t the time within an aviation context; the approach doesn’t work well in highly dynamic situations. Discussion: Where would a Classical Decision Making approach be applicable in the aviation context? 2.5 Naturalistic Decision Making Naturalistic Decision Making (NDM) perspective is based upon some key ideas: • • The information processing requirements of rational decision making exceeded limited human cognitive capacity. Humans a have to use heuristics to reduce cognitive load and speed up decision making. As a result decision makers often seek satisfactory solutions rather than trying to make optimal decisions. Human beings have bounded rationality. When pursuing a satisfactory decision, a full review of alternatives is not made. The review of options continues only until a ‘good enough’ solution is identified (one that is satisfactory or sufficient) The NDM research paradigm investigates how skilled operators use their experience to make decisions in the operational context. Unlike CDM, observations of expert decision makers indicates that they do not generate several alternatives and compare between them using pre-determined criteria to evaluate the likelihood of a successful outcome. Instead, decision makers: • • • • Focus is on understanding the situation and judging its familiarity Work sequentially from the most plausible option generated If the initial strategy is unsuitable the next most plausible option is then be generated. Decision makers use their previous experience to frame the current situation. When a good match is between the current situation and past experience, then that course of action is pursued. NDM Puts emphasis on the investigation of the processes that underlie how an individual actually makes decisions. It does not really consider the outcome. NDM is characterised by: • ‘Dynamic and continually changing conditions, real-time reactions to these changes, ill-defined tasks, time-pressure, significant consequences for mistakes’ (Klein and Klinger, 1991) 2.6 Characteristics of Naturalistic Environments Orasanu and Connolly (1991) identified eight common characteristics of naturalistic environments Coventry University: Academic Year 2015/2016 © Coventry University 2015 Module M13AAE: Decision Making and Error in Aviation • • • • • • • • Ill structured problems Uncertain dynamic environments Shifting, ill-defined or competing goals Action/feedback loops Time stress High stakes Multiple players Organisational goals and norms Exercise: Write down some examples of where these characteristics apply in your work environment and what types of decision they impact on (i.e. what sorts of thing are you making decisions about) 2.7 Decision Making Heuristics Our decision making is also affected by “rules of thumb” or heuristics that we use to expedite our decision making processes. These include: • Representativeness: The tendency to judge someone or something according to how characteristic it appears to be of a particular category of event • Availability: The tendency to consider an instance or event as being more likely if it can be easily imagined • Anchoring: The tendency to give more weight to evidence that is consistent with the decision makers initial hypothesis, rather than to any contrary information which may subsequently be gathered Exercise: Consider how the availability heuristic might impact on nervous flyers’ decisions to fly, based on what they see about safe arrivals and accidents in the media. 2.8 The NDM Process Model Naturalistic decision making often follows a two stage process: Situation Assessment, which comprises: • • • Problem Definition Risk Assessment Time Estimation Followed by Action Choice 2.8.1 Orasanu’s NDM Decision Taxonomy Orasanu (1993) developed a taxonomy of NDM decision types as shown in Figure 1. The six decision types are: Coventry University: Academic Year 2015/2016 © Coventry University 2015 Module M13AAE: Decision Making and Error in Aviation • • • Go, no-go decisions – where an action is anticipated or in progress and a cue triggers a decision to terminate that action Recognition-primed decisions – where the decision makers interpret a cue pattern as being of a particular type and match it with an action according to standardised procedures Option/response selection decisions – these occur when there are several legitimate options/actions which the decision maker needs to evaluate in terms of the requirements of the problem and from which one option needs to be selected Figure 1 NDM Decision Taxonomy (Orasanu, 1993) • Resource management decisions – in which the decision maker needs to co-ordinate several time-consuming tasks so their products are available when required (prioritisation and task allocation) • Procedural management decisions – which are typically required when a decision maker faces ambiguous high risk situations requiring a diagnosis of the problem but can then subsequently act according to standardised procedures • Creative problem solving – requires the decision maker to first diagnose and define a problem. They then need to create an action to meet the needs of the situation since no specific guidance is available These decision types are of particular interest as training can be targeted to foster each of these types of decision making, as we will see later. Coventry University: Academic Year 2015/2016 © Coventry University 2015 Module M13AAE: Decision Making and Error in Aviation Discussion: Reflect on your decision making activities over the last few days and identify which types of decisions you have been making. Post 3 examples on the discussion forum which illustrate the types of decision that you make most often, and see how they compare those of other course members. 2.9 Rasmusen’s Decision Ladder Jenkins, Stanton , Salmon and walker (2008) developed a model of decision making based upon Rasmusen’s decision ladder, underpinned by a consideration of Rasmusen’s Skills, Rules and Knowledge taxonomy of human behaviour. Skill-based behaviour proceeds without conscious attention (think of how you drive a car or perform any other well practised task). Rule-based behaviour operate on a range of IF ..THEN rules stored in long term memory, such as IF Traffic Light is on red THEN stop the car…..or perhaps IF you see a speed camera at the side of the road THEN slow down! These are available to consciousness. However, when a problem does not map directly onto a stored rule, knowledge based behaviour occurs to develop novel solutions to potentially complex problems. For example, imagine arriving at the “magic roundabout” in Swindon (Figure 2) in in the UK for the first time: how do you get from one side to the other or turn right? Figure 2 The Magic Roundabout in Swindon Coventry University: Academic Year 2015/2016 © Coventry University 2015 Module M13AAE: Decision Making and Error in Aviation Having crossed this roundabout on many occasions, It is not uncommon to see people stranded at one of the peripheral mini roundabout junctions, trying to work out what to do next. Rasmusen’s decision ladder is shown in Figure 3. Figure 3 Rasmusen’s (1974) Decision Ladder • • • • • • • Problem detection – recognising that something has happened and that some action may be required Interpretation of information – gathering relevant data and information about the situation and evaluating it in context Sensemaking – fusing the available data and information to diagnose current and future system states Identifying options – developing viable options Evaluation of competing goals – determining the priority of competing goals Selection of an appropriate task – to achieve the chosen goal Development of a procedure – to undertake the chosen task Coventry University: Academic Year 2015/2016 © Coventry University 2015 Module M13AAE: Decision Making and Error in Aviation 2.10 Improving Decision Making A number of approaches to decision making training are possible: Simulator-Based Training People make poorer decisions when under stress but need practice when under stress. Simulation offers a number of possibilities: • Training for pattern recognition by presenting simulated situations based around scenarios • Providing scenarios that occur both frequently or very infrequently – situations that would never normally experience Structuring Decision Making This is a common approach which uses mnemonic-based methods. These methods do require practice to become familiar with and use appropriately. Some examples of mnemonics are: SHOR • Stimuli - data: gather data; filter it; aggregate it and store or recall it • Hypotheses - perception alternatives: create hypotheses about the situation; evaluate them; select one • Options - response alternatives: create response options; evaluate them; select an option • Response - action: plan, organise and execute the response FORDEC • Facts – What is actually going on here? • Options – What are the choices we’ve got? • Risk and Benefits – What is there to be said for and against the application of the different options? • Decision – What shall we do? • Execution – Who shall do what, when and how? • Check – Is everything still all right? DECIDE • • D - Detect change : Are there serious risks if no action is taken? E - Estimate the significance : Are there serious risks if only the most available alternative or protective action is pursued? Coventry University: Academic Year 2015/2016 © Coventry University 2015 Module M13AAE: Decision Making and Error in Aviation • • • • S - Set safe objectives (beware hazardous attitudes): Is there a realistic possibility of finding a better solution? I - Identify options: Is there sufficient time to make a careful search for further gathering and evaluation of information and advice? D - Do the best option E - Evaluate the outcome and continue to apply the DESIDE model if there are further changes or if the decision made is not producing the desired result. Focussing training around Orasamu’s decision taxonomy Caird-Daley, Harris, Bessell and Lowe (2007) suggests the following ways in which training can be focussed for each of the decision types in the Orasanu (1997) decision taxonomy: Decision Type Training Focus  Develop perceptual patterns in memory that constitute the conditions for aborting an action  Conduct under realistic time pressure and including borderline cases  Develop recognition of situational patterns constituting the condition side of a condition-action rule  Learn the response/action side of the rule and the link between condition and action  Develop evaluation skills (what will happen if I take/don’t take this action, or is there a reason not to take this action)? Option/Response Selection Decisions  Train crew to use heuristics e.g. Representativeness; Anchoring; Availability Resource Management Decisions  Acquire knowledge of the time required to complete various tasks, and the interdependencies among tasks  Develop scheduling strategies Procedural Management  Develop situation assessment and risk assessment skill Creative Solving  Develop situation assessment and risk assessment skill  Develop skill in goal setting, planning, strategising and evaluation (e.g. case based reasoning involving presenting many examples of other’s experiences) Go/No-Go Decisions Recognition-Primed Decisions Problem- Coventry University: Academic Year 2015/2016 © Coventry University 2015 Module M13AAE: Decision Making and Error in Aviation 2.11 Decision Errors All errors are some form of decision error, but…Don’t get caught up with the outcome of the decision – aviation is a probabilistic environment ‘there is often no clear standard of ‘correctness’ [. . . and] there is a loose coupling of event outcome and decision process so that outcomes cannot be used as reliable indicators of the quality of the decision’ Orasanu and Martin (1998) Good decisions can lead to bad outcomes (and vice versa). There are two major Types of decision Error: • Error in situational assessment :Misinterpreting or ignoring cues • Error in action selection - will vary according to the type of decision strategy or • • • Errors involving rule-based decisions might depend on failing to retrieve a response from memory Errors involving decisions where a choice is required amongst alternatives may be a product of failing to retrieve an appropriate response from memory or determining the adequacy of the outcomes of the options derived Creative decisions can be prone to error as a result of the absence of any support, requiring the decision-maker to develop a novel solution • lack of experience on the part of the decision-maker • a lack of information • inadequate mental simulation Exercise: What types of decision errors (if any ) have you made this week? 2.12 FURTHER RESOURCES Suggested reading Chapter 5 of Harris, D. (2011) closely follows these notes and provides amplification. The Decision Making chapter in Wickens et al (2014) is also worth reading – it has good sections on heuristics and the factors affecting decision making performance which I commend to you. Cook et al (2007) has a host of chapters on a diverse range of current topics in decision making in complex environments which you might find interesting. Chapter 4 on risk in decision making is a good read. . Coventry University: Academic Year 2015/2016 © Coventry University 2015 Module M13AAE: Decision Making and Error in Aviation 2.13 Reading references Harris, D. (2011) Human Performance on the Flight Deck, Ashgate Wickens, D, Lee, J., Liu, Y. Gorden Becker, s. (2014) Introduction to Human Factors Engineering: Pearson New International Edition -, Pearson, Harlow Cook, M. Noys, J. Masakowski, Y (2007) Decision Making in Complex Environments, Ashgate, Aldershot. Coventry University: Academic Year 2015/2016 © Coventry University 2015