Untitled Quiz

Fundamentals of Data Science

• Course Title: DS302
• Instructor: Dr. Nermeen Ghazy

Reference Books

• Data Science: Concepts and Practice, by Vijay Kotu and Bala Deshpande (2019)
• DATA SCIENCE: FOUNDATION & FUNDAMENTALS, by B. S. V. Vatika, L. C. Dabra (2023)

Lecture 3

• No further information provided

Chapter 2: Data Science Process

• No further information provided

Modeling

• A model is an abstract representation of data and its relationships within a dataset.
• A simple rule (e.g., lower mortgage interest rates with higher credit scores) is a model.
• Modeling involves a process of creating and evaluating models, which includes splitting training and test data. (Training data is used to develop the model, test data is used to evaluate it).
• Association analysis and clustering are descriptive techniques where there's no target variable to predict. Hence, there's no test dataset for these methods.
• Both predictive and descriptive models require an evaluation step.

Application

• In business, data science results are integrated into business processes (often via software applications).
• Deployment is when the model becomes production ready.

Knowledge

• The data science process provides a framework for extracting meaningful information from data.
• To extract knowledge from large datasets, advanced data science algorithms are needed.
• The process starts with prior knowledge and ends with posterior knowledge, which is new insight gained.
• The data science process can sometimes produce spurious or irrelevant patterns.

Chapter 3: Data Exploration

• Data exploration aims to understand data structure, identify patterns, and assess data quality.
• Key tasks in data exploration include:
• Data understanding
• Data preparation
• Data science tasks
• Interpreting results

1 - Data Understanding

• Data exploration provides an overview of each attribute (variable) and interactions between attributes.
• Questions to consider during this stage include: Typical values? Variations from typical values? Extreme values?

2 - Data Preparation

• Datasets must be prepared before applying data science algorithms to address anomalies.
• Anomalies include outliers, missing values, and highly correlated attributes.
• Highly correlated attributes can negatively impact certain algorithms, so identification and removal of these attributes are crucial.

3 - Data Science Tasks

• Basic data exploration can be used as a substitute for the entire data science process (e.g., scatterplots can identify clusters).
• Data exploration can assist in developing simpler, visually based models such as regression and classification.

4 - Interpreting Results

• Data exploration aids in interpreting prediction, classification, and clustering outcomes.
• Techniques like histograms help visualize attribute distributions, making it easier to assess numeric predictions and estimate error rates.

Datasets

• The Iris dataset is a widely used dataset for learning data science.
• Iris includes 150 observations from three species (Iris setosa, Iris virginica, and Iris versicolor). Each observation has four attributes (sepal length, sepal width, petal length, and petal width), along with the species label.
• All four attributes in the Iris dataset are continuous numeric values (measured in centimeters).
• The dataset can be accessed through standard data science tools and repositories (like the UCI Machine Learning Repository).

Types of Data

• Properties of data, based on the associated operations, are different.
• Data types for example:
• Numeric (e.g., 50 cars per kilometer)
• Ordered scales (e.g., high, medium, low)
• Count of hours (e.g., number of hours with high traffic density)
  • Other types can be converted.

Descriptive Statistics

• Descriptive statistics summarize datasets to understand characteristics.
• Common applications include calculating average age, median rental prices, or determining ranges.
• Focuses on key attributes of samples or populations: Central Tendency (mean, median), Spread (range, variance), and Distribution.

Descriptive Statistics - Univariate

• Focuses on summarizing a single attribute at a time.
• Key descriptive measures:
• Measures of central tendency (e.g., mean, median, mode)
• Measures of spread (e.g., range, variance, standard deviation).

Descriptive Statistics - Multivariate

• Focuses on the relationships among multiple attributes.
• Correlation measures the statistical relationship between two attributes.

Correlation

• Correlation measures statistical relationships between attributes.
• A correlation close to +1 or -1 indicates a strong linear relationship; 0 indicates no such relationship.