Understanding Data and Data Analytics

Data vs. Information

• Data is raw, unorganized facts that need to be processed.
• Data can be simple and seemingly random until it is organized.
• Processed, organized, and structured data is called information.

Data Analytics

• Data analytics is the science of analyzing raw data to make conclusions and enhance productivity and business gain.
• Extracted data is cleaned and categorized to analyze different behavioral patterns.
• Techniques and tools used in data analytics vary according to the organization or individual.

Needs for Data Analytics

• Four main factors that signify the need for data analytics:
  • Gather hidden insights from data with respect to business requirements.
  • Generate reports from data and pass them to respective teams and individuals.
  • Perform market analysis to understand competitors' strengths and weaknesses.
  • Improve business requirements and customer experience through data analysis.

Tools in Data Analytics

• Various tools have emerged with different functionalities, including:
  • R programming: leading analytics tool for statistics and data modeling.
  • Python: provides machine learning and visualization libraries like Scikit-learn, TensorFlow, Matplotlib, Pandas, and Keras.
  • Tableau Public: free software that connects to any data source, creates visualizations, maps, and dashboards with real-time updates.
  • QlikView: offers in-memory data processing, data association, and visualization with data compression.
  • SAS: programming language and environment for data manipulation and analytics, accessible and can analyze data from different sources.
  • OpenRefine/GoogleRefine: data cleaning software for cleaning messy data, transformation, and parsing data from websites.
  • Microsoft Excel: widely used tool for data analytics, mostly used for clients' internal data, summarizes data with a preview of pivot tables.
  • RapidMiner: integrated platform that can integrate with various data sources, used for predictive analytics, data mining, text analytics, and machine learning.
  • Konstanz Information Miner (KNIME): open-source data analytics platform for analysis, modeling, reporting, and integration.
  • Apache Spark: large-scale data processing engine that executes applications in Hadoop clusters, popular for data pipelines and machine learning model development.

Data Analyst

• A data analyst is a professional who analyzes data by applying various tools and techniques, gathering required insights, and translating numbers into plain English.
• Skills required for a data analyst include:
  • Statistics
  • Data collection
  • Data cleaning
  • Data analysis
  • Hidden insights discovery
  • Data visualization
  • Reports generation
  • Machine learning

Big Data Analytics

• Big Data: a concept in Software Engineering referring to large sets of machine-generated data that are unstructured and not easy to use with traditional RDBMS concepts.
• Big Data Analytics: complex process of examining large and varied data sets to uncover information that can help organizations make informed business decisions.
• Four types of Big Data Analytics:
  • Descriptive: learning from past behaviors to understand how they might influence future outcomes.
  • Diagnostic: root cause analysis to answer the question "Why did this happen?"

Uses of Big Data Analytics

• Using Big Data Analytics to boost customer acquisition and retention, e.g., Coca-Cola's digital-led loyalty program.
• Offering marketing insights and solving advertisers' problems, e.g., Netflix's targeted advertising based on past search and watch data.
• Risk management, e.g., UOB bank's risk management system that reduces calculation time of value at risk.
• Driving innovations and product development, e.g., Amazon Fresh and Whole Foods.
• Supply chain management, e.g., PepsiCo's use of huge volumes of data for efficient supply chain management.