Business Analytics Framework Overview

Business Analytics Framework

• Focuses on a cross-industry standard process for data mining
• Aims to improve reliability and repeatability in data mining projects
• A framework for recording experience is available
• Aids in project planning and management
• Provides a "comfort factor" for new data mining adopters

The Hard Reality of Data

• Databases contain enormous amounts of data
• Businesses are often data-rich but knowledge-poor
• Data is a liability unless used to improve business practices
• Standard data analysis techniques are helpful, but insufficient and may miss valuable insights
• Popular quote by John Naisbett: "We are drowning in information, but starving for knowledge"

Examples of Data

• Transactional data from credit card companies
• Data from search engines like Google
• Social media data

What is Data Mining?

• Deployment of business processes using analytical techniques
• Aims to take further advantage of data
• Objectives include discovering relevant knowledge and acting on mining results

Data Mining Tasks

• Summarization
• Classification/Prediction (includes Classification, Concept Learning, and Regression)
• Clustering
• Dependency modeling
• Anomaly detection

Summarization

• Creates a compact description of a data subset
• Example: finding average downtime of plant equipment, total income of a sales representative
• Techniques: Statistics and Information Theory

Prediction

• Learns a function to associate data with a response variable.
• Classification learning deals with discrete variables
• Regression learning deals with continuous variables
• Example tasks: Assessing credit worthiness in loans, or predicting response to marketing campaigns
• Technique examples: Decision trees, Neural Networks, Naive Bayes

Clustering

• Identifies meaningful categories or clusters within data
• Maximizes intra-cluster similarity and minimizes inter-cluster similarity
• Example uses: Segmenting a business' customer base or creating a taxonomy of animals in a zoological application
• Example techniques: K-Means, Hierarchical clustering, Kohonen SOM

Dependency Modeling

• Discovers significant dependencies, associations, or affinities among variables
• Example uses: Analyzing market baskets in retail, or uncovering cause-effect relationships (medical treatments)
• Example techniques: Association rules, Graphical modeling

Anomaly Detection

• Discovers significant changes or anomalies in data from previous measurements or norms
• Example uses: Detecting fraudulent credit card usage or identifying anomalous turbine behavior
• Example techniques: Novelty detectors, Probability density models

Why a Standard Process?

• Provides reliability and repeatability, even for those with little data mining experience
• Facilitates project replication
• Supports project planning and management
• Simplifies the process for new adopters

CRISP-DM Framework

• Cross-Industry Standard Process for Data Mining
• An invaluable tool for planning, executing, and evaluating data mining projects
• Provides a clear roadmap for success
• Ensures well-defined, well-executed, and well-documented projects

CRISP-DM Phases

• Business Understanding
• Data Understanding
• Data Preparation
• Modeling
• Evaluation
• Deployment

Phase I: Business Understanding

• Understand project objectives and requirements
• Convert knowledge into a data mining problem definition
• Develop preliminary plans to achieve objectives
• Examples: Increase catalog sales or predict widget purchases based on customer data

Phase II: Data Understanding

• Initial data collection and familiarization
• Identify critical data quality issues
• Discover initial insights into the data
• Formulate hypotheses for hidden information

Phase III: Data Preparation

• Construct the final dataset from raw data
• Includes activities: Collection, Assessment, Consolidation, Cleaning, Data selection, and Transformations
• Includes techniques: Selecting data, cleaning data, constructive data
• May require extensive time (over 90% of the project time)

Phase IV: Modeling

• Select and apply various modelling techniques
• Generate test design
• Build the model and define parameter settings
• Assess the model and rank models based on quality and validity

Phase V: Evaluation

• Thoroughly evaluate the model and the execution steps
• Determine if business objectives are met.
• Review data mining processes, identifying and addressing any critical business issues encountered in previous phases
• Determine the next steps (project completion, new iterations, or more data acquisition)

Phase VI: Deployment

• Organize and present knowledge gained from the project
• Plan the deployment of data mining results into business processes
• Implement a detailed monitoring process to track and maintain the data mining system
• Produce final project reports; summarizing the experience, findings, and any pertinent details

Seattle Airbnb Data Analysis:

• Business Understanding - Questions related to listing prices, neighborhood distribution, and new listing price prediction based on attributes.
• Data Understanding- Descriptive statistics obtained from files such as calendar.csv, listings.csv, and reviews.csv.
• Data Preparation - techniques used to handle missing values, currency, and categorical features.
• Data Analysis - Exploration of price distribution and neighborhood listing frequency
• Data Modeling - Numerical columns related to price are analyzed using linear regression models.
• Evaluation - Evaluating model performance using metrics like R² score.
• Deployment - Implement and monitor the findings

This quiz covers the fundamental concepts of business analytics, focusing on a standardized process for data mining across industries. It explores the importance of effective data management and analysis techniques, helping organizations leverage their data for improved decision-making and knowledge gain.