Intro to Data Analytics & Data Analysis

Introduction to Data Analytics

• Data analytics is the process of collecting, organizing, summarizing, and analyzing data.
• The goal is to derive patterns, insights, and conclusions that can drive decisions and actions.
• Data transitions from raw form to actionable decisions.

Data Analysis vs Data Analytics

• Data analysis is a process of collecting, and organizing data to identify trends.
• It mainly focuses on extracting meaningful insights from data for specific purposes.
• Data analytics is a broader field that encompasses data analysis through various methodologies and technologies.
• It also involves predictive modeling and machine learning to derive actionable insights and support decision-making.
• Data analysis is more focused on historical data to answer questions like "What happened?" and "Why did it happen?"
• Data analytics takes a broader including diagnostic, predictive, and prescriptive analytics to answer different questions like "What will happen?", and "What should we do?"

Goals

• Data analysis helps to give insights into current and past data to make immediate decisions.
• Data Analytics helps generate insights for forecasts of future long-term strategies to facilitate long-term decision-making.
• Data analysis is typically used in academic studies.
• Data analytics help better understand customer preferences.
• Excel, SPSS, R and Python are common data analysis tools.
• Python, R, Tableau, Power BI, SQL, Hadoop, Spark, and cloud platforms like AWS or Google Cloud are common data analytic tools.
• Data analysis focuses on interpreting existing data to uncover patterns, trends, and insights about events that have occurred.
• Data analytics encompasses the entire process, which includes planning, analyzing, and predicting outcomes, to help guide future actions and decision-making.
• Data Analysis is a subset of Data Analytics.

Analytics Process

• The analytics process is a structured approach to solving business problems and involves several key stages

• Problem Definition

• Data Collection

• Data Preparation/Cleaning

• Data Analysis

• Interpretation of Results

• Implementation and Iteration

Problem Definition

• The foundation involves you defining the aim and scope of the analysis.
• Key questions to ask include: "What problem are you trying to solve?", "What decisions will these insights inform?", and "What to measure and how to measure it?"

Data Collection

• Crucial for gathering relevant data of objectives and data requirements.
• Key questions to ask include: "What data is needed?", "Where can the data be sourced?", "How will the data be collected?", and "Is the data reliable and relevant?"

Data Preparation/Cleaning

• Involves cleaning and organizing the data for analysis.
• Modifying data that is incorrect, duplicated or improperly formatted.
• Key questions to ask include: "What data needs to be cleaned or transformed?", "How will missing or invalid data be handled?", "How will data be standardized or formatted?", and "Are there any outliers or errors in the data?"

Data Analysis

• Focuses on analyzing data to find patterns and developing models to reach objectives
• Key activities include exploration, visualization, and modeling.
• Key Questions to ask are "What patterns or trends are present in the data?", "How can the data be visualized to highlight insights?", "What type of model or analysis is most appropriate?", "How can we validate and evaluate the findings?" and "What insights can be derived from the analysis?"

Interpretation of Results

• Entails translating analysis into actionable insights based on original problem and actionable solutions.
• Key Questions to ask are "What do the results mean in the context of the problem?", "Are the results consistent with expectations or existing knowledge?", "What are the implications of the results?", "What are the limitations or uncertainties in the results?" and "How can the results be communicated effectively?"

Implementation and Iteration

• Focuses on acting on insights on the outcomes, executing actions and monitoring the effectiveness of implemented strategies.
• Key Questions to ask are "How will the findings be applied to address the problem?", "What resources are needed for implementation?", "How will the implementation process be monitored and measured?" and "What feedback or results from the implementation are needed for iteration?"

Example of the Data Analytics Process

• Problem Definition: Understand sales trends and identify opportunities to increase revenue.
• Data Collection: Gather data from a point-of-sale (POS) system or sales tracking software.
• Data Preparation/Cleaning: Remove incorrect or incomplete records using filters.
• Data Analysis: Identify the top 10% performing products by revenue.
• Interpretation of Results: Focus marketing efforts and promote Product X with bundled discounts.
• Implementation and Iteration: Implement targeted marketing campaigns and launch a promotion for Product X, tracking sales growth and analyzing Product X sales post-campaign.

Types of Analytics

Descriptive Analytics

• Helps summarize and analyze historical data to identify patterns.
• Tracking the average number of customers visiting the store is also descriptive analytics.
• This encompasses Data aggregation/Data Summarization and Data visualization.

Diagnostic Analysis

• Examines historical data to find the reason behind outcomes.
• Investigating root causes of increased complaints in customer support is an example.
• Main techniques are Drill-down analysis/Data Decomposition and Statistical analysis(Correlation and regression).

Predictive Analysis

• Uses historical data to predict futures.
• Forecasting sales for the next quarter is an example.
• Some methods are Machine learning models, Time series analysis & Predictive modeling.

Prescriptive Analysis

• Involves recommending actions to optimize outcomes based on predictions and scenarios.
• Optimizing delivery routes to reduce shipping costs is prescriptive analysis.
• Methods used are Optimization algorithms, Decision trees and Simulation models.

Importance/Applications of Analytics

• Analytics provides evidence-based insights to make decisions, reducing reliance on intuition.
• Problem-solving using analytics involves analyzing data to recognize the root causes of issues to develop solutions.
• Analytics maximizes output while minimizing resource usage, costs, and effort by giving insights into operations, workflow,identifying inefficiencies and data driven improvements.
• Analytics allow businesses to find opportunities, anticipate trends, adapt to market demands with precision providing a competitive edge.
• Analytics plays a crucial role by providing data-driven insights of identifying potential risks and managing them.

Importance of Analytics in Specific Fields

Education

• Analytics evaluates teaching methods' to find better student outcomes.
• Helps identify struggling students early to provides special interventions.
• Guides the design of courses and learning materials based on data-driven insights into student needs

Health Care

• Uses Predictive analytics to identify high-risk patients and personalizes treatment plans.
• Manage resources like staff, equipment effectively.
• Tracks disease outbreaks and evaluates interventions through Public Health.

Business and Marketing

• Analyzes purchasing behavior to develop targeted marketing campaigns.
• Predicts demand patterns to reduce wastage and meet consumer needs through Inventory Management.
• Strategic Planning: Guides long-term strategies through market trend analysis.

Government

• Supports evidence-based policymaking by evaluating the impact of previous initiatives in Policy Development.
• Makes sure of efficient use of funds and resources for public welfare.
• Predictive analytics helps in crime trend analysis and prevention strategies through Crime Prevention.

Overview of Analytics Tools and Technologies

Microsoft Excel

• Spreadsheet software useful for data wrangling, reporting, and visualization.
• Widely-used, versatile tool with many built-in functions.
• Limitation in handling big data.

Python

• Programming language that is useful for data analysis, machine learning, and web scraping.
• It has open-source, extensive libraries (Pandas, NumPy, Matplotlib).
• Cons: Requires programming knowledge

R

• Programming language used for statistical analysis, data visualization.
• It's strong in statistical computing, open-source.
• It has a steeper learning curve compared to Python.

Microsoft Power BI

• Business analytics service that helps in Data visualization, reporting.
• Integrates with other Microsoft products, user-friendly.
• Has limited advanced analytics capabilities.

Tableau

• It's a data visualization tool which helps in interactive dashboards, data storytelling.
• Great User interface and powerful visualization capabilities.
• Can be expensive, requires training.

SPSS

• A statistical software which helps with data management, statistical analysis and reporting.
• User-friendly, extensive statistical capabilities, strong data manipulation features.
• Can be expensive, less flexible compared to some programming languages like Python and R.

KNIME

• It's an open-source analytics platform which helps in data integration, reporting, machine learning.
• It has No coding required, modular.
• Performance can be slow with large datasets.

Weka

• It's a machine learning and data mining software.
• It helps in Data preprocessing, classification, regression, clustering, association rule mining, and visualization.
• It has an extensive library of machine learning algorithms, user-friendly GUI, supports various data formats, includes tools for data cleaning and preparation.
• Can be less efficient with very large datasets, limited customization compared to some programming languages.

Analytics Tools According to the Type of Data Analytics

Descriptive Analytics

• Spreadsheets: Microsoft Excel, Google Sheets.
• Visualization Tools: Tableau, Microsoft Power BI.
• Databases: SQL for querying (MySQL, PostgreSQL).

Diagnostic Analytics

• Statistical Software: Excel Analysis ToolPak, SPSS.
• Programming: Python (pandas, matplotlib), R (ggplot2).
• Exploration Platforms: Tableau, Power BI (with drill-down capabilities).

Predictive Analytics

• Spreadsheet Models: Excel (basic forecasting with built-in tools).
• Machine Learning Libraries: Scikit-learn (Python), caret (R).
• Cloud Services: Google AutoML, AWS Forecast (easy-to-use platforms).

Prescriptive Analytics

• Optimization Tools: Solver in Excel.
• Decision Support Tools: Simple simulation models using spreadsheets.
• Cloud-Based Tools: Salesforce Einstein (integrates with business operations).