Data Analytics Foundations: Pre-Course Reading

• Data Analytics Foundations is pre-course reading for the Data Analytics Foundation course.
• The reading duration is approximately 3 hours.
• The material provides basic knowledge needed for the Foundation Course in Data Analytics.
• The material is useful for those new to data, data analysis, and basic Maths and Statistics.

Contents Overview

• Understanding Data: Types, Sources, and Importance

• The Data Analysis Process: Key Stages and Workflow

• Exploratory Data Analysis (EDA): Understanding Data Patterns

• Data Cleaning and Preprocessing: Preparing Data for Analysis

• Data Analytics refers to the end-to-end process of transforming raw data into valuable insights for decision-making.

• The process involves collecting data, cleaning and organizing it, and then applying analytical methods to uncover trends, patterns, and relationships.

Understanding Data

• Data refers to raw, unprocessed facts and figures collected from various sources.
• Data is prepared and analyzed to extract meaningful insights.
• Analyzing data is crucial because it reveals patterns or insights that are not obvious.
• Different types of analysis are: Descriptive, Predictive, Diagnostic, and Prescriptive.

Types of Analytics

• Descriptive analytics summarizes past data to understand historical trends and patterns, answering the "what" question.
• Diagnostic analytics explains why something happened by looking at the factors or reasons behind trends.
• Predictive analytics uses current and historical data to predict what might happen in the future.
• Prescriptive analytics offers actionable advice on what you should do next based on data analysis.

Structure of Data

• Structured data fits neatly into rows and columns, is highly organized.
• A spreadsheet or database table is a great example of structured data.
• Semi-structured data has some organization but doesn't follow strict rules, e.g., JSON files or XML documents
• Unstructured data is a wild mix of everything, e.g., text, images, videos, social media posts with no clear format.

Data Sources

• Common data sources are databases and surveys or forms.
• Social media platforms like Twitter or Instagram provide unstructured data.
• Emails structured data.
• Website logs generate semi-structured data.

SQL

• Databases are organized data collections in a structured format.
• SQL (Structured Query Language) is used to get information from a database.
• SQL commands are called queries.
• SELECT is used to retrieve data from a table e.g. SELECT * FROM products;.
• WHERE is used to filter data based on a condition e.g. SELECT * FROM products WHERE price > 10000;.
• ORDER BY is used to sort data e.g. SELECT * FROM products ORDER BY price DESC;.

Data Analysis Process: Key Stages and Workflow

• The data analysis process is a systematic approach that turns raw data into valuable insights.
• 1. Defining Objectives and Requirements: Identify business goals and analytical requirements.
• 2. Data Collection: Gather data from various sources, ensuring high quality and relevance.
• 3. Data Processing and Cleaning: Prepare the data by removing errors, inconsistencies, or duplicates.
• 4. Data Exploration and Analysis: Explore the data using visualizations and basic statistics to identify trends and anomalies.
• 5. Interpretation of Results: Interpret the results in the context of the organization's goals and communicate insights effectively.
• 6. Implementation and Action: Apply insights to drive action, adjust strategies, and optimize processes.
• 7. Maintenance and Iteration: Continuously maintain and update models, as data and business environments evolve.

Data Exploration

• After fully understanding the problem, the next step is understanding the data itself.
• Data exploration is the process of examining and analyzing the dataset to uncover its key characteristics, such as patterns, relationships, and anomalies.
• Data exploration ensures analysts work with relevant, accurate, and meaningful data.
• Analysts use statistics and visualization to explore the data
• Statistical techniques i.e. means, medians, standard deviations, correlations, quantify relationships and trends within the data.
• Visualization tools like histograms, scatter plots, and box plots.

Statistical Techniques

• Statistics is the branch of mathematics that deals with collecting, analyzing, interpreting, and presenting data.
• It is used to summarize data, identify patterns, and understand relationships between different variables.
• Calculates the Mean or Median.
• It can also be affected by outliers.
• Statistics gives analysts the power to draw meaningful insights from raw numbers.
• The Mean is the sum of all the numbers divided by the total number of values.
• The Median is the middle value in a dataset when all the numbers are arranged in order.
• Outlier: Is a value that is significantly different from the rest of the data.
• Standard Deviation measures of how spread out the numbers are in a dataset. A low standard deviation means the numbers are close to the mean, while a high standard deviation means the numbers are more spread out.
• Variance measures the average degree to which each number is different from the mean.

Visual exploration

• Visuals bring data to life, showing patterns, trends, and relationships quickly.
• Histograms are charts that show of how data is spread out, and where most values are located
• Box plots are used for spotting outliers, show the spread of data.
• Scatter plots are used to plot relationship between two variables.
• Line charts are used to track changes over time.
• Heatmaps represent data activity at a glance.
• Distribution charts help you understand how data points are spread across a range.

Data Cleaning and Preprocessing

• Data Cleaning involves fixing or removing errors, inconsistencies, or missing values in the dataset.
• Preprocessing prepares data so that it's in a usable format for analysis
• Some of these issues can include outliers, that requires more attention if not corrected will result to misleading conclusions
• To do that outliers must be detected and handled correctly.
• Missing Values when certain data points are not recorded, but need to be imputed. These measures need to be followed to ensure that the data can be used and can be the basis for insights.

Methods to fix Missing Data

• Remove missing data
• Imputation
• Leave them as missing

Data Visualization

• Analysts use various methods, i.e dashboards, reports, slides, and interactive visualizations to present data findings.
• A dashboard is a visual display of key metrics and data points.
• Dashboards provide real-time monitoring of data, it allows users to react quickly.

How to choose to right charts

• Comparison charts for comparing different Items.
• Bar charts displays data with rectangles.
• Line chart connects data points with lines, they can reveal great trends.
• Circular Charts show data in a circular format.
• Relationship charts explore connections.
• Composition Charts show different parts.
• Pie Chart is used well to show percentages.
• Stacked colums Shows multiple categories.
• Distribution charts Help understand how data points are spread within a category