Data-Driven Decision Making PDF

24/04/24 Master in Information Management...

24/04/24 Master in Information Management Data-Driven Decision Making Introduction to Data-Driven Decision-Making Fernando Lucas Bação [email protected] http://www.isegi.unl.pt/fbacao Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 1 Information as a Strategic Resource Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 2 1  24/04/24 Seven “Laws” of Information Information Is (Infinitely) Shareable; The Value of Information Increases With Use; Information is Perishable; The Value of Information Increases With Accuracy; The Value of Information Increases when Combined; More Is Not Necessarily Better; Information is not Depletable. Source: http://si.deis.unical.it/zumpano/2004-2005/PSI/lezione2/ValueOfInformation.pdf 3 Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 3 Seven “Laws” of Information (Moody & Walsh ECIS 99) 1st – Information Is (Infinitely) Shareable Unique characteristic; it can be shared between business areas or organizations without any loss of value; Most assets are appropriable, either you have it or you don’t; Sharing it tends to multiply its value; Infinitely replicated; Balkanization of information with multiple versions and lack of consistency; Duplicating information does not double its value. Source: http://si.deis.unical.it/zumpano/2004-2005/PSI/lezione2/ValueOfInformation.pdf 4 Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 4 2  24/04/24 Seven “Laws” of Information (Moody & Walsh ECIS 99) 2nd – The Value of Information Increases With Use Information has no value on its own - it only becomes valuable when one uses it; It is essencial that organizations know the information they have, only then can it be used to create value; Few organizations have a catalogue of the information they own; Tha ability to improve the decision-making process by the increase of information depends on the decision-maker's information literacy. Source: http://si.deis.unical.it/zumpano/2004-2005/PSI/lezione2/ValueOfInformation.pdf 5 Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 5 Seven “Laws” of Information (Moody & Walsh ECIS 99) 3rd – Information is Perishable Like most other assets, the value of information tends to depreciate over time; There are several types of information with associated levels of useful life: Operational information; Decision Support Information/Analitycs; Statutory information. The rise of data warehouses for storing historical information without operational relevance to power decision support and analysis systems. Source: http://si.deis.unical.it/zumpano/2004-2005/PSI/lezione2/ValueOfInformation.pdf 6 Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 6 3  24/04/24 Seven “Laws” of Information (Moody & Walsh ECIS 99) 4th – The Value of Information Increases With Accuracy The more accurate information is, the more useful and valuable it is; Inaccurate information can be very costly to an organization (both operationally and in terms of decision- making); Below a certain level becomes a liability. Source: http://si.deis.unical.it/zumpano/2004-2005/PSI/lezione2/ValueOfInformation.pdf 7 Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 7 Seven “Laws” of Information (Moody & Walsh ECIS 99) 5th – The Value of Information Increases When Combined Information tends to become more valuable when compared, combined or integrated with other information; The integration and consolidation of information from different operating systems is a premise in building analytical capabilities; Typically one of the most complex, costly and critical tasks. Source: http://si.deis.unical.it/zumpano/2004-2005/PSI/lezione2/ValueOfInformation.pdf 8 Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 8 4  24/04/24 Seven “Laws” of Information (Moody & Walsh ECIS 99) 6th – More Is Not Necessarily Better In most cases, the more of a certain resource you have, the better off you are; how to allocate limited resources is a core management problem; The problem today is not the lack of information but the overabundance of it; Evidence shows that our comprehension degrades rapidly when we exceed a certain limit; Human decision-makers tend to seek more information than can be optimally processed, in an effort to reduce uncertainty. Source: http://si.deis.unical.it/zumpano/2004-2005/PSI/lezione2/ValueOfInformation.pdf 9 Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 9 Seven “Laws” of Information (Moody & Walsh ECIS 99) 7th – Information is not Depletable Most resources: the more you use, the less you have it; Information is self-generating, the more you use it, the more you have; Information is not a scarce resource. Source: http://si.deis.unical.it/zumpano/2004-2005/PSI/lezione2/ValueOfInformation.pdf 10 Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 10 5  24/04/24 Information as a Strategic Resource (sources) Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 11 Information as a Resource (sources) § Objective of collection Primary Secondary § Organization Structured Unstructured § Origin Internal External 12 Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 12 6  24/04/24 Information as a Resource (sources) § Primary data refers to data that are collected with a particular analysis objective in mind. Typically, primary data results from a specific inference question. In this way the datasets are collected with this specific objective in mind and according to well established methodological directives, in accordance with the specific needs of the inference task. § Secondary data is related with data that were collected with some other purpose but can also be used to perform analysis. Secondary data results from different types of digital processing and operational systems, which, in the majority of the cases, are not concerned with inference. 13 Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 13 Information as a Resource (sources) § Primary data: Expensive; High quality; Limited to an objective; § Secondary data: Inexpensive; Low quality; Generic; 14 Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 14 7  24/04/24 Information as a Resource (sources) § Structured data refers to any data that resides in a fixed field within a record or file. This includes data contained in relational databases and spreadsheets. Structured data first depends on creating a data model – a model of the types of business data that will be recorded and how they will be stored, processed and accessed. This includes defining what fields of data will be stored and how that data will be stored: data type (numeric, currency, alphabetic, name, date, address) and any restrictions on the data input (number of characters; restricted to certain terms such as Mr., Ms. or Dr.; M or F). Structured data has the advantage of being easily entered, stored, queried and analyzed. 15 Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 15 Information as a Resource (sources) § Unstructured data usually refers to information that doesn't reside in a traditional row-column database. It's the opposite of structured data — the data stored in fields in a database. Unstructured data files often include text and multimedia content. Examples include e-mail messages, word processing documents, videos, photos, audio files, presentations, webpages and many other kinds of business documents. Note that while these sorts of files may have an internal structure, they are still considered "unstructured" because the data they contain doesn't fit neatly in a database. Experts estimate that 80 to 90 percent of the data in any organization is unstructured. And the amount of unstructured data in enterprises is growing significantly — often many times faster than structured databases are growing. 16 Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 16 8  24/04/24 Information as a Resource (sources) § Structured data refers to information with a high degree of organization, such that inclusion in a relational database is seamless and readily searchable by simple, straightforward search engine algorithms or other search operations; § Unstructured data is essentially the opposite. The lack of structure makes compilation a time and energy-consuming task. 17 Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 17 Information as a Resource (sources II) § Internal data: Internal data are those which are generated and obtained from the information systems of an organization. Information that is produced by the organization in its activity and that it is essential to monitor their activity. § External Sources: External data refer to the information collected by third parties. Market information and enables the organization to understand the environment in which it operates, it can be about the competition or the consumers. 18 Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 18 9  24/04/24 Information as a Resource (sources II) § Internal information allows: understanding the level of performance; seek new efficiencies through innovation; increase the stability of the customer base through customer satisfaction thus enhancing their loyalty. § Today we are witnessing a huge development in the areas that produce organization, exploration and presentation tool for organizing internal information organizations. 19 Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 19 Information as a Resource (sources II) § External information allows : foresee opportunities and risks related to the organizations activity; increasing competitiveness, in particular as regards the creation of new markets and products; It is an important component of innovation processes. 20 Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 20 10  24/04/24 Creating Sustainable Competitive Advantage Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 21 Creating Sustainable Competitive Advantage § The Basic Argument Information infrastructure of a company is an asset, a property of the company, which by definition is owned by one company and not the other. The company can use its information to serve its customers better by identifying the correct services to offer, make product recommendations, or tailor promotions more effectively than its competition can do with this set of customers. It can also use this information to improve its processes and efficiency. This asymmetric information gives a company a potential sustainable competitive advantage. 22 Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 22 11  24/04/24 Information as a Resource (sources) § The Basic Argument It is sustainable because it would cost the competition too much to obtain the same information – they would have to buy the company. Increasingly the value of a company is determined by the value of its customer file The information infrastructures are proprietary and their advantage grows as the company learns from them and improves its customer offerings even more. 23 Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 23 Information as a Resource (sources) § Evolution of the Sustainable Competitive Advantage Argument The Customer Information File as a Firm Asset Glazer (1991, 1999) – Information from transactions with suppliers – Information from internal operations – Information from transactions with customers 24 Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 24 12  24/04/24 Information as a Resource (sources) § Evolution of the Sustainable Competitive Advantage Argument Customer information contributes in three ways (Glazer 1991, 1999): increased revenues from future transactions (e.g., through better targeting of the right products at the right price), reduced costs (e.g., through not having to mail every offer to every customer), and the sale of information itself (through say renting the customer list). 25 Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 25 Information as a Resource (sources) § Evolution of the Sustainable Competitive Advantage Argument The Customer Information File as a Firm Asset Rust et al. (2002) take a related but somewhat different perspective. They conceptualize the choice as between revenue expansion (focus on the customer), and cost reduction (focus on decreasing operations and organizational costs). They find that firms perform better when they focus on revenue expansion, illustrating the importance of customer information, than when they focus on both revenue expansion and cost reduction. 26 Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 26 13  24/04/24 Information as a Resource (sources) § Evolution of the Sustainable Competitive Advantage Argument The Customer Information File as a Firm Asset Glazer echoes Moorman’s (1995) point that markets that are driven by customer information, the ability to process information, not the information itself, is the scarce resource. Thus, the source of competitive advantage to a firm is a combination of creating customer information, processing of the information and then utilizing the information to drive superior marketing strategies. 27 Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 27 Information as a Resource (sources) § New business models and sources of revenue 28 Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 28 14  24/04/24 Data-Driven Decision-Making Context Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 29 Agenda Data-Driven Decision-Making Context Single Version of the Truth Infobesity Timely decision making Data network effects Less Intuition, More Evidence Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 30 15  24/04/24 Single Version of the Truth Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 31 Single Version of the Truth Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 32 16  24/04/24 Single Version of the Truth Too much worthless data, instead of a gold mine we have a lot of junk (GIGA); A data rich and information poor environment: Difficulty in finding the relevant information; Multiple truths about key indicators: Absence of semantic standardization, "the beginning of wisdom, is the definition of terms" Socrates; Lack of integration, Balkanized information (found in silos); Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 33 Single Version of the Truth Dependent Analysis and Delivery Data M art Source Systems Data Data Warehouse Data Integration Access Dependent Data M art Metadata Metadata Processes Data Quality Processes Governance Processes Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 34 17  24/04/24 Single Version of the Truth Dependent Analysis and Delivery Data M art Source Systems Data Data Warehouse Data Integration Access Dependent Data M art Metadata Metadata Processes Data Quality Processes Governance Processes Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 35 Infobesity in Decision-Making Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 36 18  24/04/24 Single Version of the Truth Lack of sensitivity to the aspects related with the access and analysis costs: All information is "interesting"; We ask for queries and reports without regard to their cost; At the end of the day they seldom serve for decision making; Difficulty in Processing: The sheer volume of available information can exceed an individual's capacity to process it effectively, leading to a decrease in comprehension and retention. Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 37 Timely decision making Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 38 19  24/04/24 Timely decision making Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 39 Timely decision making Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 40 20  24/04/24 Data Network Effects Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 41 Data Network Effects Network effects Firms of yesteryear - petrochemicals, cars, chemicals, steel, and so on - were relatively stable oligopolies where a few companies dominated an industry, temporary monopolies are what is observed in the world of high tech. The difference is in the economics of scale, versus the economics of networks and positive feedback, which reinforces the strong, and hurts the weak. This tends to create one or a few very strong winners, and few other companies in a market. 42 Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 42 21  24/04/24 Data Network Effects Network effects The “old” companies worked with supply-side economies of scale - whoever could grow the most could cut costs and grow profits because of the efficiencies created by their massive size. Information companies instead work with demand side economies of scale: people value a product because a lot of other people use it. This creates a virtuous cycle for winners - since a lot of people use your product, even more will decide that they need to as well. Products with fewer users lose those users to products with more users, falling into a vicious cycle. 43 Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 43 Data Network Effects Network effects The concept of network effect is well understood: A flywheel type situation where a good or service becomes more valuable when more people use it. Many examples out there from the telephone system (the value of a phone increases if everyone has a phone) to Facebook to many marketplaces (with some nuances for the latter). “type situation where a good or service becomes more valuable when more people use it” 44 Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 44 22  24/04/24 Data Network Effects Data Network effects Data network effects occur when your product, generally powered by machine learning, becomes smarter as it gets more data from your users. The more users use your product, the more data they contribute; the more data they contribute, the smarter your product becomes (which can mean anything from core performance improvements to predictions, recommendations, personalization, etc. ); the smarter your product is, the better it serves your users and the more likely they are to come back often and contribute more data – and so on and so forth. Over time, your business becomes deeply and increasingly entrenched, as nobody can serve users as well. 45 Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 45 Data Network Effects “Data network effects occur when your product, generally powered by machine learning, becomes smarter as it gets more data from your users.” 46 Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 46 23  24/04/24 Data Network Effects “Waze is another example, essentially a contributory database built on data network effects.” 47 Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 47 Less Intuition, More Evidence Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 48 24  24/04/24 Less Intuition, More Evidence Intuition Intuition works well only in specific environments, where there are good clues and the subject receives almost immediate feedback (e.g. firefighters and poker players); Intuition takes time to be built (10 years in the case of chess players); The cognitive biases are several (42 related to decision- making) and perform a decisive influence on our decisions. Cognitive Biases: http://www.businessinsider.com/cognitive-biases-that-affect-decisions-2015-8?utm_source=feedly Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 49 Less Intuition, More Evidence Some Academic Research: The criteria used by psychologists to diagnose their patients (drawn from psychologists) was used to created a model; Comparison between those psychologists and the model in new patients… the model failed less than the experts that created it; The advantage is that a model has no intuition (or moods!) Fonte: “Man versus model of man: A rationale, plus some evidence, for a method of improving on clinical inferences.” By Goldberg, Lewis R., Psychological Bulletin. Vol 73(6), Jun 1970, 422-432. Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 50 25  24/04/24 Less Intuition, More Evidence Some Academic Research: Evaluation of 136 studies that compared the performance of human and analytical models In 65 studies, no significant differences were found; In 63, it was concluded that the models had a significantly better performance; In 8, humans were shown to be able to produce more accurate predictions. MODELS 46% - humans 6% Fonte: “Clinical versus mechanical prediction: A meta-analysis.” By Grove, William M.; Zald, David H.; Lebow, Boyd S.; Snitz, Beth E.; Nelson, Chad, Psychological Assessment. Vol 12(1), Mar 2000, 19-30. Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 51 Less Intuition, More Evidence “intuition is similar to what I think of Tom Cruise’s acting ability: real, but vastly overrated and deployed far too often” Andrew McAfee Fonte: https://hbr.org/2010/01/the-future-of-decision-making Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 52 26  24/04/24 Less Intuition, More Evidence “Weak human + machine + better process was superior to a strong computer alone and, more Software remarkably, superior to a strong Hardware human + machine + inferior process.” Processes and Garry Kasparov Governance Analytical Talent Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 53 Less Intuition, More Evidence Impact in decision making processes: New decision-making processes First, they can get in the habit of asking “What do the data say?” “Where did the data come from?,” “What kinds of analyses were conducted?,” “How confident are we in the results?” Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 54 27  24/04/24 Less Intuition, More Evidence Impact in decision making processes: Muting the HiPPOs; Traceability of results; Collective decision-making; Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 55 Exercise 1 - Let’s do a predictive model!!! Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 56 28  24/04/24 Exercise 1 - Let’s do a predictive model!!! I’m the owner of the Wonderful Wines of the World (WWW) company. I sell wines to my customers on a subscription model, based on a website. Often, whenever I come across an item related with wine and that can appeal to wine connoisseurs like my customers, I buy a large quantity and try to sell it to my database of customers. Presently, I have 10.000 customers on my database and given that I have data scientists working for me I decided that, for this promotion (cork extractor) instead of sending the brochure to the entire database I’ve send it to just 2.000 customers, as a test. Now I’ve received their responses and know which were the customers interested in the item (a cork extractor). Now I need you to select from the 8.000 still in the database the ones that have the highest probability of buying the cork extractor. Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 57 Exercise 1 – Model assessement I need to know: 1. Which and how many clients should I send the proposal to? 2. What’s the benchmark for the model? 3. What response rate should I expect? 4. Estimate of how much I will earn, taking into account that each shipment costs me €3 and with each purchase I earn €15. Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 58 29  24/04/24 Questions? Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa 59 59 30