Introduction to Analytics Lesson 4 PDF

INTRODUCTION TO ANALYTICS 2022 - 2023  LESSON 4. ADVANCED ANALYTICS, MACHINE LEARNING & AI Learning Objectives 1. Understand the meaning of prediction 2. Recognize predictor and response variables 3. Discuss data used for advanced analytics 4. Distinguish type of predictive models 5. Di...

INTRODUCTION TO ANALYTICS 2022 - 2023  LESSON 4. ADVANCED ANALYTICS, MACHINE LEARNING & AI Learning Objectives 1. Understand the meaning of prediction 2. Recognize predictor and response variables 3. Discuss data used for advanced analytics 4. Distinguish type of predictive models 5. Distinguish between ML and AI 6. Explain the two types of AI 7. Ask advanced analytics questions Agenda 1. What is prediction? 2. From prediction to decision 3. Predictor and response variables 4. Data used in advanced analytics 5. Types of predictive models 6. Advanced analytics questions 7. ML and AI; two types of AI 8. Group assignment introduction  LESSON 4. WHAT IS PREDICTION ?  Types of Advanced Analytics Descriptive The examination of data or content, to answer the question “What Analytics happened?” or “What is happening?” Diagnostic A form of advanced analytics that examines data or content to answer analytics the question, “Why did it happen?” Advanced Analytics Predictive A form of advanced analytics which examines data or content to Analytics answer the question “What is going to happen?” or more precisely, “What is likely to happen?” Prescriptive A form of advanced analytics which examines data or content to Analytics answer the question “What should be done?” or “What can we do to make X happen?” https://www.gartner.com/en/information-technology/glossary Types of Analytics Advanced Determine the best Analytics course of action Infer why has it Predict and forecast occurred future Monitor what has occurred What is prediction? Prediction Prediction is about using information you have to generate information you don't have Harvard Business Review Predict: foretell on the basis of observation, experience, or scientific reason https://www.merriam-webster.com/dictionary/predict In science, a prediction is a rigorous, often quantitative, statement, forecasting what would happen under specific conditions https://en.wikipedia.org/wiki/Prediction#Prediction_in_science  Predictive Modeling We solve business problems to: Improve efficiency Make better decisions Enable the organization do something new that was not possible before In predictive modeling, we take a business problem and reframe it as a prediction problem: What is someone likely to do in a given scenario? What is likely to happen if…? How much …. is expected when….? Value of Prediction How can a prediction help solve a business problem?  Goal of analytics Analytics: The examination of information to uncover insights that give a businessperson the knowledge to make informed decisions. Actionable insights Intel Developer Works Community  Predictive Modeling Examples Pipe breakdown prediction: https://www.cnbc.com/advertorial/pipeline-to-success-saving-trillions- through-japanese- innovation/?utm_source=linkedin&utm_medium=cpv&utm_campaign =CP1&3&utm_content=G7fracta OR https://www.linkedin.com/posts/activity-6640523504376745984-YN78  Predictive Modelling Examples Survival analysis - models that predict occurrence and timing of the events e.g. customer churn (attrition), patient survival, customer's next purchase, loan default Network analysis - target particular individuals within a network at persons for advertising: optimise advertising budget by targeting the centre of the network Net lift response modelling - predict the likelihood of customers responding to a promotion / campaign FROM PREDICTION TO DECISION Prediction vs Decision  Decision can be… Automated based on scores from the models Partially automated (flow through) for standard behaviours, but allow override rules to handle exceptional cases. E.g. Use the model to make predictions for majority of cases. Not automated - require humans judgement using the scores and predictions provided by the model. First, make a prediction Then, make a decision based on prediction (automated or not) What do humans do with analytics?  Analytics maturity 1. How much of the decision process is automated? 2. How much humans intervention in decision making process is required? Advanced Analytics  PREDICTOR AND RESPONSE VARIABLES  Predictive Modelling Model: a mathematical representation of an object or a process. Predictive modelling: using mathematical and computational methods to develop predictive models that examine datasets for underlying patterns and calculate probability of an outcome Predictive modelling is about understanding the relationships between Predictor data and Behavioral (Response) data. Predictive analytics process: analyses the data to identify how the predictor data can be used to differentiate between behaviours and predict outcomes.  Predictor and Response Variables Predictor variable: the variable used to predict the response a.k.a Independent variable Feature Attribute X-variable Response variable: the variable that the model is aiming to predict a.k.a Dependent variable Outcome Target Y-variable Example https://www.kaggle.com/c/titanic/data  Example Dependent variable ~ Response variable Predictor variables ~ independent variables https://www.kaggle.com/c/titanic/data Practice: Predictor vs Response variables Work in groups: A streaming platform has collected a lot of data about user behaviour on the platform, as well as customer surveys. The goal is to improve the platform movie recommendations algorithm. What prediction would you recommend? What prediction and response variables can you think of? Mini-quiz Predictor vs Response variables  DATA USED IN ADVANCED ANALYTICS  Sources of predictor variables Source What to look for Existing metadata Existing documentation that describes business data, relationships, and constraints Current decision- What data is currently used to make business decisions. making process What parameters and attributes are included in business reports and operational analytics that management uses to make decisions today? Subject matter experts People that are involved in day-to-day business process will know what information working in operational is important and relevant areas External data sources External sources such as credit rating agencies or business associations have rich knowledge of relevant data and its importance for prediction Research / peer Investigate what data is used my industry, competitors and peers for similar business experience problems Steven Finlay (2014) Predictive Analytics, Data Mining and Big Data (Business in the Digital Economy) Palgrave Macmillan  Data Used for Advanced Analytics Data Type Description Primary Information about the past behaviour that is of the same type as behaviour you behaviours want to predict. Example: committed a burglary before Secondary Information about past behaviours that are similar (but different) to the one you behaviours want to predict. Example: minor crimes and offenses Tertiary Information about past behaviours that do not have an obvious connection to behaviours the behaviour you want to predict. Example: selling items on eBay Steven Finlay (2014) Predictive Analytics, Data Mining and Big Data (Business in the Digital Economy) Palgrave Macmillan  Data Used for Advanced Analytics Data Type Description Geo- Information about a person’s state of being demographic Example: age, income, occupation, appearance, education level Associate Information about person’s connections (their geo-demographic data and data behaviours) Example: partner or friends that committed a burglary in the past Sentiments A person’s attitudes, feelings and opinions Example: social media likes/dislikes, approvals/disapprovals, comments, blog posts, survey answers Network Information about the nature of connections between a person and their associates Example: majority of friends have committed criminal offences Steven Finlay (2014) Predictive Analytics, Data Mining and Big Data (Business in the Digital Economy) Palgrave Macmillan  TYPES OF PREDICTIVE MODELS  Machine Learning Supervised learning Unsupervised learning Predict outcomes using Predict outcomes using labeled data unlabeled data Learn from labeled data to detect patterns. Model the underlying structure or distribution in the data in order to learn A set of data (called “training set”) is more about the data. available where correct answers, or “response variables” (“labels”) are Data is unlabeled (i.e. correct answer known for given scenarios , or “input is unknown) variables”  Machine Learning Supervised learning Unsupervised learning Predict outcomes using Predict outcomes using labeled data unlabeled data Clustering Association Classification models Regression models find a structure or Predict the probability pattern in a Discover of the event Predict a quantity relationships (estimate value) collection of (estimate probability) uncategorized data between variables Supervised Learning: Predictive Models Model Type Description Example Classification Predict the probability Probability that an email Logistic regression (binary problems) problem: of the event (from a is Spam/Not Spam Decision tree Classify an discrete list of events / Probability that a Random forest instance possibilities); customer will place a Naive Bayes Output: probability in new order within next Support Vector Machine (SVM) % month Neural networks Response variable is Nearest neighbour categorical Regression Predict a quantity or Estimate a Customer Linear regression problem: amount Lifetime value (CLV) - Polynomial regression Find a function Output: estimate total amount of money a Ridge regression that fits the (numeric value) customer is expected to Decision tree data with the Response variable is spend in your business, or Random forest least error numerical on your products, during Neural networks their lifetime Nearest neighbour https://www.guru99.com/unsupervised-machine-learning.html ADVANCED ANALYTICS QUESTIONS How to ask predictive analytics questions? How to ask predictive analytics questions? What is someone likely to do in a given scenario? What is likely to happen if…? How much …. is expected when….? What is the expected for …? What is the likelihood that someone will choose …. if…? What is the probability that …. belongs to …. category? MACHINE LEARNING AND ARTIFICIAL INTELLIGENCE Is machine learning and artificial intelligence the same thing? Where does deep learning fit in? menti.com Machine Artificial Leaning Intelligence (ML) (AI) Deep Learning (DL)  Machine Learning vs. AI Artificial intelligence (AI) is the theory and development of computer systems able to perform tasks normally requiring human intelligence, such as visual perception, speech recognition, decision-making, and translation between languages. — Oxford Reference Machine learning (ML) is an artificial intelligence (AI) discipline that allows computers to handle new situations via analysis, self-training, observation and experience. — https://www.techopedia.com/ Deep Learning (DL) is a subfield of machine learning concerned with algorithms inspired by the structure and function of the brain called artificial neural networks. — Machine Learning Mastery  Computer Science Artificial Intelligence (AI) Machine Leaning (ML) Deep Learning (DL) Source: Shubhendu and Vijay (2013; p. 29)  Types of AI Artificial Intelligence (AI) General AI Narrow AI Intended to be capable of a Intended for a narrowly broad set of tasks defined or single task comparable to the tasks that can be done by a human https://www.techopedia.com/  Narrow vs. General AI starwars.com Narrow AI General AI Goal Intended to perform a single task Intended to perform a broad set of tasks that a human is capable of. Principle Process and analyze data, build Replicate or imitate human cognitive prediction models and make process quantitative predictions Problems Solves problems that are framed as Intended to solve any problem that a prediction problems human can solve AKA Weak AI Broad / Strong / Full AI Examples All AI that exists today is narrow See fiction books; research phase  Narrow AI Intended for a narrowly Analytics Applications Examples defined or single task Recommender systems - based on previous search results for a user Targeted advertising - targeted based on user’s past behavior; social media analytics Speech recognition – sound patterns are compared to find best match; predict intention of speech Typing autocomplete – predicting the words based on the context and first letters Gaming - algorithms which improve / upgrade themselves as the player moves up to a higher level Fraud and risk detection – likelihood to default on loan is calculated based on customer profile Logistics - best routes to ship, best time to deliver, best mode of transport Sports cameras – real-time prediction of game direction and switching cameras  Later in the program Module 7: Analytics Managing analytics projects project basics Roles in analytics projects Quantitative Research Descriptive statistics Methods course Probability and probability distributions, hypothesis testing Big Data course Loading, storing and processing big data Querying big data Big data manipulation, analysis and visualization Data Analytic Tools Traditional and advanced analytics tools course Data manipulation Quantitative analysis and statistical techniques Final thought Data Analytics Prediction Decision Action Google Research The unreasonable effectiveness of data: The size of the training dataset may matter more to the success of the prediction than the choice of the prediction model.