Data Analyst Specialist Lecture 3 PDF

Data Analyst Specialist - Lecture 3 1 Introduction to Problem Solving 2 What is Problem Solving? Problem-solving in data analysis refers to a methodical approach used to understand, address, and resolve business issues using data. It is crucial for...

Data Analyst Specialist - Lecture 3 1 Introduction to Problem Solving 2 What is Problem Solving? Problem-solving in data analysis refers to a methodical approach used to understand, address, and resolve business issues using data. It is crucial for businesses as it helps in making informed decisions, optimizing processes, and driving innovation. 3  Step-by-Step Problem-Solving Approach DEVELOPING IMPLEMENTING AND ANALYZING GATHERING DATA - SOLUTIONS - EVALUATING INFORMATION - USE IDENTIFYING THE PROBLEM - COLLECT DATA FORMULATE SOLUTIONS - APPLY ANALYTICAL METHODS CLEARLY DEFINE THE ISSUE AT RELATED TO THE STRATEGIES OR THE CHOSEN TO PROCESS THE HAND. PROBLEM FROM ACTIONS BASED ON SOLUTION AND DATA AND UNCOVER VARIOUS SOURCES. THE INSIGHTS ASSESS ITS INSIGHTS. GAINED. EFFECTIVENESS. 4 DMAIC Framework Overview The DMAIC framework is a systematic problem-solving method used for improving, optimizing, and stabilizing business processes and designs. It includes: 1. Define: Articulate the problem and specify the customer requirements. 2. Measure: Quantify the problem and collect relevant data. 3. Analyze: Investigate and determine the root cause of the defects. 4. Improve: Develop and implement solutions to address the root causes. 5. Control: Sustain the improvements through controls and monitoring. 5  DMAIC Example Problem 1: High Defect Rates in Manufacturing 1.Define: Clearly state the problem—high defect rates in a specific product line. 2.Measure: Collect data on current defect rates and identify when and where defects occur most frequently. 3.Analyze: Analyze the collected data to find patterns or common factors in defect occurrences. 4.Improve: Develop solutions such as adjusting machine settings, improving worker training, or modifying the material used. 5.Control: Implement the changes and continuously monitor defect rates. Establish regular checks and audits to maintain low defect rates. 6  DMAIC Example Problem 2: Customer Complaints About Call Center Efficiency Define: Identify the main issue—customers are complaining about long waiting times. Measure: Measure the average handle time, waiting time, and call drop rates. Analyze: Determine the reasons for long waiting times, possibly due to staffing issues, inefficient call routing, or inadequate training. Improve: Implement solutions like optimizing staffing schedules, upgrading call routing software, or providing additional training for staff. Control: Regularly review call center performance metrics to ensure improvements are sustained. Adjust strategies as needed based on ongoing data analysis. 7 PDCA (Plan-Do-Check-Act) The PDCA cycle emphasizes a cyclical approach to continuous improvement: 1. Plan: Identify and analyze the problem, develop hypotheses about potential issues, and plan possible solutions. 2. Do: Implement the planned solution on a small scale to test its effect. 3. Check: Monitor and evaluate the effectiveness of the solution against the expected results. 4. Act: If the solution was successful, implement it on a wider scale and continuously assess the results. If the solution did not work, revisit the plan and begin the cycle again. 8  PDCA Example Problem 1: Software Feature Underperformance Plan: Identify the Act: If the improvements underperforming feature. are effective, roll out the Hypothesize reasons for changes to all users. If underperformance, such not, analyze the feedback as user interface and start another PDCA complexity or lack of cycle with a different necessary functionality. approach. Do: Implement changes Check: Measure user on a small scale, such as engagement with the new simplifying the interface version of the feature and or adding requested gather user feedback. features. 9  PDCA Example Problem 2: Inconsistent Sales in Retail 1.Plan: Plan to enhance staff training and modify store layouts to promote higher-priority products. 2.Do: Trial the new training program in one store and rearrange the layout as planned. 3.Check: Monitor sales results in the trial store and compare them to sales in stores that did not undergo the changes. 4.Act: If sales improve, implement the changes across all stores. If there is no improvement, reassess the situation and plan a new set of changes. 10 Root Cause Analysis (RCA) Root Cause Analysis is a systematic process used to identify the underlying causes of faults or problems. It involves the following steps: 1. Identify the Problem: Clearly define the problem or fault that occurred. 2. Collect Data: Gather data and evidence related to the problem to form a complete picture. 3. Analyze Data: Analyze the data to identify patterns or issues that point to potential causes. 4. Identify Root Causes: Use tools like the Five Whys or fishbone diagrams to drill down to the root cause(s) of the problem. 5. Develop Solutions: Propose solutions to address the root causes. 6. Implement and Monitor: Implement the solutions and monitor the results to ensure the problem is resolved. 11  RCA Example Problem 1: Frequent Downtime in IT Systems 1.Identify the Problem: Recognize that IT system downtime is frequent and affecting business operations. 2.Collect Data: Gather data on each incident of downtime, noting times, durations, and conditions. 3.Analyze Data: Look for commonalities among incidents. Use tools like the fishbone diagram to categorize potential causes. 4.Identify Root Causes: Apply the Five Whys method to each category to drill down to the root causes—e.g., outdated hardware or software conflicts. 5.Develop Solutions: Propose solutions such as upgrading hardware or resolving software conflicts. 6.Implement and Monitor: Execute the solutions and monitor system stability to ensure the problem is resolved. 12  RCA Example Problem 2: High Employee Turnover 1.Identify the Problem: Note that employee turnover rates are higher than industry averages. 2.Collect Data: Gather and analyze data from exit interviews, employee satisfaction surveys, and HR reports. 3.Analyze Data: Identify patterns or common reasons cited for leaving. 4.Identify Root Causes: Use the Five Whys method to explore reasons behind dissatisfaction, such as inadequate career development opportunities or unsatisfactory management practices. 5.Develop Solutions: Design programs to improve career development opportunities, provide training for managers, or enhance work-life balance. 6.Implement and Monitor: Implement these strategies and regularly review turnover rates and employee satisfaction to assess the effectiveness of the solutions. 13 Comparison Summary DMAIC PDCA RCA focuses intensely on facilitates quick, iterative is highly structured, suits the underlying causes learning and adjustments, complex problems where in- and is best for ideal for dynamic Definition depth analysis and long-term addressing specific, environments that require control are needed. recurring, or critical agility. problems. Applicable in any Suitable for both new and Best suited for projects aimed setting where problems existing processes. Often at improving existing occur, especially useful used in environments that processes, where the process in addressing recurring require iterative testing and Use Cases is already in place but not problems or where a agile adjustments, such as performing to the desired solution to a single manufacturing and software level of quality or efficiency. problem can have development. broad implications. Focuses on Encourages experimental Highly structured, data- eliminating the root learning and quick driven, focused on statistical cause of problems adjustments; ideal for analysis, and aims at rather than just Strengths environments that need sustainable, long-term addressing flexible and iterative improvements. symptoms; can 14 testing. prevent future  Case Study 1: Improving Restaurant Wait Times (DMAIC Method) Background: A popular local restaurant is experiencing longer- than-average wait times during peak hours. This issue is leading to increased customer complaints and a noticeable loss in business. The restaurant seeks to reduce wait times to improve customer satisfaction and increase overall profitability. 15  Case Study 2: Software Upgrade Rollout (PDCA Method) Background: A technology firm is preparing to roll out a major software update designed to enhance user experience and introduce new features. Previous upgrade rollouts have encountered user resistance and bug-related complaints, affecting user satisfaction and the overall success of the updates. The firm aims to use the PDCA method to ensure a smoother rollout, minimize issues, and improve user acceptance. 16  Case Study 3: Reducing Employee Turnover (Root Cause Analysis) Background: A retail company has been experiencing high employee turnover over the past year, especially among new hires within their first six months of employment. This high turnover rate is impacting team stability and increasing recruitment and training costs. The company aims to use Root Cause Analysis to identify the underlying causes of this issue and develop strategies to improve employee retention. 17 CRAFT EFFECTIVE QUESTIONS 18  Things to Avoid When Asking Questions : 1. Leading Questions 5. Overly Complex Questions Definition: Questions that suggest a particular Definition: Questions that combine multiple response. topics. Example: "Don’t you think our new product is Example: "How do you feel about our product’s great?" features, pricing, and overall value?" 2. Closed-Ended Questions 6. Multiple Questions at Once Definition: Questions that elicit only brief Definition: Asking more than one question in a responses. single query. Example: "Do you like the new feature? Yes, or Example: "What do you think about our new No?" product’s design and functionality?" 3. Vague Questions Definition: Questions that are too broad or lack context. Example: "How do you feel about our service?" 4. Ambiguous Questions Definition: Questions that can be interpreted in multiple ways. Example: "What do you think about the changes?" 19 SMART QUESTIONS 20  Crafting SMART Questions for Effective Data Analysis: Specific: Ensure the question is targeted and provides clear direction. Example: Instead of "How is our customer base growing?", ask "What is the monthly growth rate of new customers over the past 12 months?" Measurable: The responses to the question should be quantifiable. Example: Instead of "Is our marketing campaign effective?", ask "What is the conversion rate of our marketing campaign, and how does it compare to the previous campaign?" Action-Oriented: The question should prompt or guide actionable outcomes. Example: Instead of "What are our customer satisfaction scores?", ask "What specific changes can we make to our customer service processes to improve our satisfaction scores by 15%?" Relevant: Directly relate to the problem at hand. Example: Instead of "What are the most common customer complaints?", ask "Which customer complaints are most frequently mentioned in relation to our latest product launch, and how can addressing these improve customer satisfaction?" Time-bound: Focus the inquiry on a specific time frame. Example: Instead of "How are our sales performing?", ask "What was our total sales revenue for Q2 of this year compared to Q2 of last year?" 21  Learn to Ask the Right Questions in Data Analytics: The SMART Questioning Framework: SMART: Specific, Measurable, Action-oriented, Relevant, Time-bound. Purpose: Ensures questions are constructed to yield valuable, actionable insights. Examples of SMART Questions: Specific and Measurable: Transform vague questions into targeted inquiries. Before: Are kids active enough? After: What percentage of kids engage in physical activity for 60 minutes, five days a week? Relevant and Time-Bound: Focus the inquiry on specific scenarios and time frames. Example: Investigate environmental changes from 1983 to 2004 affecting Pine Barrens tree frogs in Durham, NC. 22  Learn to Ask the Right Questions in Data Analytics: Action-Oriented Questioning: Questions should drive actionable insights and lead to meaningful changes. Example: Instead of asking if customers like the packaging, ask what features would make the packaging easier to recycle. Ensuring Fairness in Questions: To avoid bias and ensure questions are inclusive and neutral: Avoid leading respondents with assumptions. Implement clear, unbiased phrasing that allows for honest and diverse responses. 23 Understanding Data in Decision- Making 24  Understanding Data in Decision-Making 1. Data is a raw form of information before processing. 2. Data analysis identifies patterns and insights. 3. Insights from data lead to informed decisions. 25  Types of Decision-Making 1.Data-Driven Decisions: Decisions made solely based on data analysis, with minimal to no influence from intuition or subjective judgment. 2.Data-Inspired Decisions: Decisions influenced by data but also consider other factors such as experience, intuition, and context. 26  Data-Driven Decisions Definition: Data-driven decision-making (DDDM) refers to decisions that are based exclusively on data analytics and quantitative analysis. Decisions are made purely on what the data suggests without input from personal experience or intuition. Characteristics: Objective and Quantifiable: Decisions are made based on statistical data and metrics, reducing the influence of biases and assumptions. Consistency: Using data exclusively can lead to consistent decision-making, as decisions are standardized based on data models and metrics. Reliance on Technology: Heavily depends on data analytics tools, algorithms, and comprehensive data sets. Lack of Flexibility: May not fully consider qualitative factors like human emotions or cultural impacts, which are difficult to quantify and include in data models 27  Data-Driven Decisions Advantages: Reduces Risk: Lower risk of errors in decision-making by eliminating guesswork and subjective judgment. Scalable: Easy to scale across an organization as it relies on set metrics and models that can be applied universally. Efficiency: Streamlines decision-making processes, often automating them to enhance speed and efficiency. Disadvantag Data Quality Dependency: Highly dependent on the quality and es: completeness of data; poor data can lead to misguided decisions. Overlook Contextual Factors: Might overlook non-quantifiable factors that could be crucial for some decisions. Potential for Overfitting: In complex scenarios, relying strictly on data might result in models that are overfit to historical data, potentially failing under different conditions. 28  Data-Driven Decisions Example Definition: Directly use data to guide business strategies, though decisions are limited by the quality and quantity of the data available. Example: A/B Testing – Testing different website layouts to determine which one increases widget sales, based on visitor data. 29 Data-Inspired Decisions Definition Data-inspired decision-making involves using data as a key input among other factors, such as intuition, experience, or external conditions. Decisions are influenced by data but not dictated solely by it. Characteristics: Balanced Approach: Combines hard data with personal or collective expertise and contextual understanding. Adaptability: More flexible, allowing for adjustments based on factors outside of what the data can show. Inclusive of Qualitative Insights: Integrates qualitative data and subjective factors into the decision-making process. 30 Data-Inspired Decisions Holistic View: Provides a more rounded approach to decision-making, considering both quantitative data and qualitative insights. Flexibility: Adapts more easily to changes and new information, including external market conditions or internal organizational changes. Adva Enhanced Creativity: Allows for innovative solutions by not being strictly ntag bound to what historical data indicates. es: Risk of Bias: Greater risk of subjective biases affecting decisions, as personal judgments and experiences play a role. Less Predictable: Can lead to inconsistencies in decision-making, as different people might interpret the influence of data differently. Disa Complexity in Justification: It can be more challenging to justify decisions dvan purely on empirical grounds, as subjective factors are harder to quantify and tage defend in some business contexts. s: 31  Data-Inspired Decisions Example Blend data with other inputs such as intuition and experience to make decisions. Example: Customer Support Center – Using customer satisfaction scores (CSAT) and qualitative feedback to improve customer service. 32 CHALLENGES IN DATA ANALYTICS 33 34  Challenges in Data Analytics 1. Data Quality and Integrity 2. Data Integration and Silos Issue: Poor quality data, including Issue: Integrating data from various inaccurate, incomplete, or inconsistent sources and breaking down data data, can lead to misleading analysis silos within an organization can be results. Ensuring data integrity is often technically complex. Data may be challenging due to the vast amounts of stored in different formats or data collected from diverse sources. systems that are not readily Impact: Decisions made based on faulty compatible. data can lead to significant financial loss, Impact: Lack of integration can damaged reputation, and misguided prevent organizations from obtaining business strategy. a holistic view of their operations or customers, limiting the effectiveness of analytics. 35  Challenges in Data Analytics 3. Data Privacy and Security 4. Lack of Skilled Personnel Issue: With increasing regulatory Issue: There is a high demand for skilled requirements, such as GDPR and CCPA, data scientists, analysts, and engineers, and growing public concern over privacy, but a significant skills gap remains in the ensuring data is handled securely and market. This shortage can hinder an ethically is paramount. organization's ability to implement and Impact: Non-compliance can lead to leverage data analytics effectively. hefty fines and a loss of consumer trust, Impact: Without the right talent, which can be devastating for businesses. organizations may struggle to interpret data correctly and leverage it for strategic decision-making. 36  Challenges in Data Analytics 5. Cost of Implementation 6. Managing Big Data Issue: Setting up robust data analytics Issue: The sheer volume and velocity of infrastructure requires significant data generated today can overwhelm investment in technology and human traditional data processing applications. resources. Small to medium-sized Impact: Organizations may struggle to enterprises (SMEs) may find the cost process, analyze, and extract value from prohibitive. big data in real time or near real time, Impact: High costs can delay or prevent which is often necessary for timely the adoption of data analytics practices, decision-making. putting businesses at a competitive disadvantage. 37  Challenges in Data Analytics 7. Cultural Resistance 8. Ethical Concerns and Bias Issue: Integrating data-driven decision- Issue: Algorithms and data sets can making into a company’s culture can face inadvertently contain and propagate resistance from employees accustomed to biases, leading to ethical concerns, traditional decision-making processes. particularly in sensitive areas like hiring Impact: Resistance can slow down or practices or loan approvals. derail the adoption of data analytics Impact: Biased analytics can lead to initiatives, limiting their potential benefits. unfair outcomes and discrimination, potentially causing legal and reputational issues. 38  Challenges in Data Analytics 10. Complexity in 9. Maintaining Relevance Visualization Issue: The rapid pace of change in both Issue: As data grows more complex, so technology and business environments does the task of visualizing it effectively. means that data and analytics processes Creating intuitive and insightful need to be continually updated to remain visualizations requires both technical skill relevant. and artistic sense. Impact: Organizations that fail to keep Impact: Poorly designed data their data analytics strategies aligned with visualizations can lead to misinterpretation current technologies and practices may of data or overlook critical insights. lose out to more agile competitors. 39 QUALITATIVE AND QUANTITATIVE DATA 40 41  Quantitative Data: Definition: Specific, objective, and measurable numerical facts. Examples: Number of commuters taking the train weekly. Used in Visualized through charts and graphs for clarity and precision. Data Collection Methods: Structured interview Survey Poll 42  Qualitative Data: Definition: Describes qualities or characteristics that are not numerically measurable. Examples: Reasons behind preferences for a celebrity or snack food. Itis important to Provide context to numerical data, answering "why" questions. Data Collection Methods: Focus group Social media text analysis In-person interview 43  Combining Data Types in Business: Case Study: Local Ice Cream Shop Using Customer Reviews Quantitative Analysis: Count of negative reviews. Average ratings. Frequency of specific keywords. Qualitative Analysis: Understanding reasons behind customer dissatisfaction, e.g., frustration due to lack of popular flavors. 44  IMPORTANCE OF VISUALIZING AND SHARING DATA EFFECTIVELY. 45  Understanding Reports: Definition: Static collections of data provided to stakeholders periodically. Reports are best for periodic high-level data reviews. Benefits: Offers snapshots of historical data (e.g., monthly sales). Easy to design and reference. Utilizes cleaned and sorted data. Drawbacks: Requires regular maintenance. Not visually dynamic; does not reflect live data changes. Example: Monthly new followers count. 46  Exploring Dashboards: Definition: Tools that monitor and display live, incoming data. Dashboards are suited for continuous, detailed data interaction. Benefits: Interactive and dynamic, offering real-time data insights. Saves time by reducing the need for repetitive report generation. Visually appealing and provides extensive data access. Drawbacks: Time-consuming to design. Requires maintenance if base tables break. Can overwhelm users with complex data sets. Example: Live tracking of engagement across multiple platforms. 47