Data Attributes and Missing Values Quiz
44 Questions
0 Views

Choose a study mode

Play Quiz
Study Flashcards
Spaced Repetition
Chat to Lesson

Podcast

Play an AI-generated podcast conversation about this lesson

Questions and Answers

Which attribute type includes values with a meaningful order but unknown magnitude between successive values?

  • Discrete
  • Ordinal (correct)
  • Nominal
  • Binary

Which of the following is an example of a ratio-scaled attribute?

  • Temperature in Celsius
  • Weight in kilograms (correct)
  • Hair color
  • Army rankings

What best describes a binary attribute?

  • A nominal attribute with only two states. (correct)
  • An attribute with three or more possible values.
  • An attribute that can take on any numeric value.
  • An attribute that is always measurable on an ordered scale.

In which of the following scenarios would you use an interval-scaled attribute?

<p>Recording temperatures in Fahrenheit. (D)</p> Signup and view all the answers

Which of the following is NOT a characteristic of nominal attributes?

<p>Values can be ordered or ranked. (B)</p> Signup and view all the answers

What is a common issue with manually filling in missing values?

<p>It is tedious and often infeasible. (A)</p> Signup and view all the answers

Which of the following methods is considered smarter for filling in missing values?

<p>Using the attribute mean for all samples belonging to the same class. (A)</p> Signup and view all the answers

What causes noise in data?

<p>Faulty data collection instruments. (D)</p> Signup and view all the answers

Which of the following is NOT listed as a problem associated with data?

<p>Excessive documentation. (C)</p> Signup and view all the answers

What is the purpose of binning in handling noisy data?

<p>To reduce cardinality and smooth the data. (B)</p> Signup and view all the answers

What does the cosine similarity formula calculate?

<p>The similarity between two vectors (C)</p> Signup and view all the answers

What is a common issue encountered when integrating multiple databases?

<p>Data redundancy (A)</p> Signup and view all the answers

Which operation is used to calculate the dot product of two vectors?

<p>Multiplying corresponding elements of the vectors (C)</p> Signup and view all the answers

Which of the following is NOT a method of data transformation?

<p>Data validation (C)</p> Signup and view all the answers

What is the significance of calculating the lengths of vectors in cosine similarity?

<p>To normalize the vectors for accurate comparison (B)</p> Signup and view all the answers

What issue do traditional similarity measures face?

<p>They fail to capture complex semantics (C)</p> Signup and view all the answers

What does the min-max normalization formula achieve?

<p>It maps values to a new specified range (B)</p> Signup and view all the answers

Which of the following is an attribute that may vary across different databases?

<p>Derived attribute (C)</p> Signup and view all the answers

How is the cosine similarity value represented mathematically?

<p>As the dot product divided by the product of the magnitudes of the two vectors (D)</p> Signup and view all the answers

What method can be used to detect redundant attributes within datasets?

<p>Correlation analysis (A)</p> Signup and view all the answers

Which example illustrates the limitation of vector space models?

<p>Both previous sentences convey the same meaning (D)</p> Signup and view all the answers

Which of the following describes 'smoothing' in data transformation?

<p>Removing noise from data (D)</p> Signup and view all the answers

What new methods are suggested for handling complex semantics?

<p>Distributive representation and representation learning (C)</p> Signup and view all the answers

What is the primary purpose of discretization in data transformation?

<p>To categorize continuous data (D)</p> Signup and view all the answers

What fundamental aspect do data quality and data cleaning focus on?

<p>Maintaining integrity and consistency of data (B)</p> Signup and view all the answers

What does attribute construction involve in data transformation?

<p>Creating new attributes from existing ones (D)</p> Signup and view all the answers

What does a boxplot represent regarding a dataset?

<p>The quartiles and outliers of the dataset (A)</p> Signup and view all the answers

What is the inter-quartile range (IQR)?

<p>The difference between the 75th percentile and the 25th percentile (A)</p> Signup and view all the answers

What does a quantile-quantile (q-q) plot compare?

<p>The quantiles of one distribution with those of another (D)</p> Signup and view all the answers

Which statement about positively and negatively correlated data is true?

<p>One half can be positively correlated while the other is negatively correlated. (B)</p> Signup and view all the answers

Which of the following describes an ordinal variable?

<p>A variable where the order of values is significant, like rankings (D)</p> Signup and view all the answers

What is the correct threshold for identifying potential outliers in a dataset using IQR?

<p>1.5 x IQR (B)</p> Signup and view all the answers

In a scatter plot, what does each point represent?

<p>A pair of coordinates representing two variables (B)</p> Signup and view all the answers

What can cosine similarity be used to measure?

<p>The frequency of terms in a document (C)</p> Signup and view all the answers

Which of the following is NOT part of the five-number summary?

<p>Range (B)</p> Signup and view all the answers

How is the distance between ordinal values calculated?

<p>By using the rank of the values (A)</p> Signup and view all the answers

What is a key limitation when using equal-frequency partitioning for data presentation?

<p>It does not handle skewed data well. (C)</p> Signup and view all the answers

Which method is employed in smoothing data using the binning approach?

<p>Replacing all values in a bin with the bin's mean. (D)</p> Signup and view all the answers

What is a common challenge when managing categorical attributes in data processing?

<p>They can complicate data partitioning. (A)</p> Signup and view all the answers

What characterizes lossless compression in data handling?

<p>It ensures that the original data can be perfectly reconstructed. (A)</p> Signup and view all the answers

Which of the following is true regarding audio/video compression?

<p>It generally employs lossy techniques for refinement. (C)</p> Signup and view all the answers

What does the term 'bin boundaries' refer to in the context of data smoothing?

<p>The nearest values used to represent bins. (C)</p> Signup and view all the answers

What happens to outliers when using straightforward data representation methods?

<p>They can dominate the presentation of the data. (B)</p> Signup and view all the answers

What is the result of using equal-depth partitioning?

<p>All intervals contain approximately the same number of samples. (C)</p> Signup and view all the answers

Flashcards

Nominal Attribute

Categorical data where values represent distinct groups or states with no inherent order. Examples include hair color, marital status, and zip codes.

Binary Attribute

A nominal attribute with only two possible states, typically represented by 0 and 1. Examples include gender or a medical test result (positive/negative).

Ordinal Attribute

A type of attribute where the order of values matters, but the difference between them is not defined. Examples include grades (A, B, C), army rankings, or size categories (small, medium, large).

Interval Attribute

A type of attribute with equal interval sizes where values have order, but there's no true zero point. Examples include temperature in Celsius or Fahrenheit, and calendar dates.

Signup and view all the flashcards

Ratio Attribute

A type of attribute with a true zero point, allowing for ratio comparisons. Examples include temperature in Kelvin, length, counts, and monetary quantities.

Signup and view all the flashcards

Noise

Random errors or variations in measured data.

Signup and view all the flashcards

Causes of Noisy Data

Incorrect attribute values can be caused by faulty data collection tools, data entry mistakes, transmission errors, or limitations in technology.

Signup and view all the flashcards

Global Constant for Missing Values

Replacing missing values with a globally constant value, such as 'unknown' or a new class.

Signup and view all the flashcards

Binning

The practice of grouping data into bins to reduce the number of unique values and smooth out irregularities.

Signup and view all the flashcards

Smoothing Data using Binning

A technique for handling noisy data by smoothing values within bins using means, medians, or bin boundaries.

Signup and view all the flashcards

Cosine similarity

A mathematical way to measure the similarity between two vectors d1 and d2, often representing term-frequency vectors. It is calculated by taking the dot product of the two vectors and dividing by the product of their magnitudes.

Signup and view all the flashcards

Data Preprocessing

The process of transforming raw data into a format suitable for analysis. It often involves cleaning, transforming, and reducing data, enabling improved accuracy and meaningful insights.

Signup and view all the flashcards

Data Cleaning

A type of data preprocessing that involves identifying and correcting errors, inconsistencies, or missing values within the dataset. It aims to ensure data quality and reliability for further analysis.

Signup and view all the flashcards

Data Integration

The process of combining data from multiple sources into a unified dataset. Integration addresses inconsistencies, duplicates, and data anomalies, creating a cohesive and consistent view.

Signup and view all the flashcards

Vector Space Model

A technique that uses a vector space model to represent words or documents as mathematical vectors, allowing for computational comparison of their semantic similarity. It involves measuring the angle between these vectors in a multi-dimensional space.

Signup and view all the flashcards

Hidden Semantics

The limitations of traditional vector space models that often fail to capture the nuanced meanings and complexities of language. They are often unable to differentiate between words with similar surface structures but distinct semantic interpretations.

Signup and view all the flashcards

Distributive Representation

A technique that emphasizes the semantic relationships between words, going beyond simple word counts. It creates distributed representations of words, capturing the contextual and relational aspects of language.

Signup and view all the flashcards

Representational Learning

A field in machine learning focused on learning meaningful representations of data, typically in the form of vectors. It aims to extract and encode the essence of information from complex data structures, enabling more accurate and efficient analysis.

Signup and view all the flashcards

Object Identification

A technique used in data integration to address situations where the same attribute or object has different names across multiple databases. The goal is to ensure consistency and avoid duplication.

Signup and view all the flashcards

Derivable Data

A situation where an attribute can be derived from other attributes in a table, creating redundancy. For example, calculating age from date of birth.

Signup and view all the flashcards

Redundancy in Data Integration

The problem arising from redundant data, potentially leading to inconsistencies, storage inefficiencies, and difficulties in data analysis. Redundant attributes may be detected by correlation or covariance analysis.

Signup and view all the flashcards

Normalization

A data transformation technique that aims to reduce the range of values in an attribute by scaling them to a smaller, specified range. Common methods include min-max normalization, z-score normalization, and normalization by decimal scaling.

Signup and view all the flashcards

Discretization

A data transformation method that converts continuous data into discrete intervals. This can help simplify analysis, improve visualization, and reduce noise.

Signup and view all the flashcards

Data Compression

The process of reducing the size of a dataset without losing significant information. Techniques include lossy compression, which removes some data, and lossless compression, which maintains all data.

Signup and view all the flashcards

Sampling

A data transformation approach that involves selecting a subset of data from a larger dataset, typically to reduce the size of the data or to make analysis more manageable.

Signup and view all the flashcards

Data Transformation

A function that maps a set of attribute values to a new set of values, preserving the relationship between original and transformed values. This can involve various techniques like smoothing, attribute construction, aggregation, normalization, or discretization.

Signup and view all the flashcards

Scatter Plot

A graph that shows the relationship between two variables by plotting pairs of values as points in a plane. Each point represents a data pair, and the position of each point on the plane indicates the values of the two variables.

Signup and view all the flashcards

Quantile-Quantile Plot (Q-Q Plot)

A method for graphically comparing the distributions of two datasets by plotting their quantiles against each other. It helps visualize if the two datasets follow similar distributions or deviate significantly.

Signup and view all the flashcards

Boxplot

A visual representation of data distribution using a box and whiskers. It shows the median, quartiles (Q1, Q3), and the range of the data, along with potential outliers.

Signup and view all the flashcards

Inter-Quartile Range (IQR)

A statistical measure that represents the difference between the third quartile (Q3) and the first quartile (Q1). It indicates the spread of the middle 50% of the data.

Signup and view all the flashcards

Five Number Summary

A set of five key statistical measures that summarize the distribution of data. It includes the minimum, first quartile (Q1), median, third quartile (Q3), and the maximum.

Signup and view all the flashcards

Outliers (in Box Plot)

Values that fall outside a specific range, typically 1.5 times the interquartile range (IQR) below Q1 or above Q3. These values are often considered unusual or extreme.

Signup and view all the flashcards

Ordinal Variable

A type of variable where the order of values is meaningful, but the differences between them may not be equal. Examples include ranks, grades, and opinion scales.

Signup and view all the flashcards

Bag of Terms Representation

A way to represent documents or other objects as vectors, where each element (attribute) corresponds to the frequency of a specific feature (like a word or keyword) in that object.

Signup and view all the flashcards

Positively Correlated Data

Data that shows a positive relationship between two variables. As one variable increases, the other also tends to increase.

Signup and view all the flashcards

Equal-depth Partititoning

A data preprocessing technique where the data is divided into equal-sized intervals or bins. Each bin contains roughly the same number of data points.

Signup and view all the flashcards

Smoothing by Bin Means

A data smoothing technique where the values within a bin are replaced with a single representative value, like the bin's mean or boundary. This reduces noise and makes the data smoother.

Signup and view all the flashcards

Smoothing by Bin Boundaries

A data smoothing technique where the values within a bin are replaced with the closest bin boundary value. This is another way to reduce noise and smooth the data.

Signup and view all the flashcards

Lossless String Compression

A type of data compression where the original data can be fully reconstructed from the compressed data. This is a lossless compression method, meaning no information is lost.

Signup and view all the flashcards

Lossy Audio/Video Compression

A type of data compression where the original data cannot be fully reconstructed from the compressed data. This is a lossy compression method because some information is lost during compression.

Signup and view all the flashcards

Progressive Refinement

A technique used in data compression that breaks down the signal into smaller fragments and reconstructs the original signal based on these fragments. It is often efficient but can also lead to quality loss.

Signup and view all the flashcards

Fragment Reconstruction

In audio/video compression, data is often compressed in fragments without reconstructing the whole signal. Smaller parts can be retrieved for processing without the need for rebuilding the entire file.

Signup and view all the flashcards

Study Notes

Data, Measurements, and Data Preprocessing

  • This chapter covers various types of data sets and methods for preprocessing data.
  • Data types include relational records, data matrices (e.g., numerical matrices, crosstabs), transaction data, and document data (represented as term-frequency vectors).
  • Graphs and networks are also mentioned, including transportation networks, World Wide Web, molecular structures, and social/information networks.
  • Ordered data is presented, including video data (sequences of images), temporal data (time series), sequential data (transaction sequences), and genetic sequence data.

Types of Data Sets

  • (1) Record Data: Relational records—highly structured relational tables, data matrices, crosstabs are included
  • Various examples of data, like records for persons and cars in a data set are shown, including a data matrix example.
  • (2) Graphs and Networks: Transportation network, World Wide Web, molecular structures, and social/information networks are discussed.
  • (3) Ordered Data: Video data, temporal data (time-series), sequential data (transaction sequences). Genetic sequence data are examples.
  • (4) Spatial, Image, and Multimedia Data: Spatial data (maps, vector and raster representations) and image and video data are included.

Data Objects

  • Data sets consist of data objects.
  • Each data object represents an entity.
  • Examples given are sales databases (customers, items, sales), medical databases (patients, treatments) and university databases (students, professors, courses)
  • Data objects are characterized by attributes.
  • Database rows represent data objects, while columns represent attributes.

Attributes

  • Attributes are characteristics or features/variables of a data object.
  • Examples are customer ID, name, and address.
  • Attribute types include nominal (e.g., red, blue), binary (e.g., true, false), ordinal (e.g., freshman, sophomore), numeric (quantitative), interval-scaled, and ratio-scaled.
  • Discrete attributes have finite or countably infinite values; continuous attributes have real-valued attribute values.

Attribute Types

  • Nominal: Categorical values (e.g., hair color, marital status)
  • Binary: Two states (0/1 or true/false) - symmetric (equal importance) or asymmetric (unequal importance).
  • Ordinal: Values with a meaningful order (e.g., grades, rankings).
  • Numeric: Quantitative values; further categorized as:
    • Interval-scaled: Differences between values are meaningful (e.g., Celsius temperature, calendar dates)
    • Ratio-scaled: Values have a true zero point (e.g., Kelvin temperature, length, counts)

Statistics of Data

  • Measures of central tendency (mean, median, mode) and dispersion are important for understanding the data distribution
  • Covariance and correlation analysis reveal the relationship between numerical variables.
  • Graphs display basic statistical descriptions like boxplots, histograms, quantile plots, scatter plots.

Measuring the Central Tendency

  • Mean: The average value (algebraic measure, sample vs. population)
  • Weighted arithmetic mean: Multiplies each value by its corresponding weight.
  • Trimmed mean: Removes extreme values.
  • Median: The middle value. Estimable by interpolation on grouped data.
  • Mode: The most frequent value.

Measuring the Dispersion of Data

  • Quartiles: Q₁ (25th percentile), Q₃ (75th percentile), and the Interquartile Range (IQR).
  • Five-number summary: Min, Q₁, median, Q₃, and Max.
  • Boxplots: Graphical representation of the five-number summary.
  • Outliers are discussed.

Measuring Data Distribution

  • Variance: A measure of the spread of data points around the mean.
  • Standard deviation: The square root of variance.
  • Formulas for sample and population variances/standard deviations are included.

Correlation

  • Correlation measures the linear relationship between two numerical variables.
  • It's computed as the normalized covariance between the variables.
  • Correlation values range from -1 to 1; values near 1 indicate positive correlation; near -1 indicates negative correlation; values near 0 indicate no correlation.

Visualizing Changes of Correlation Coefficients

  • Scatter plots can visualize correlation coefficients varying between -1 and 1.

Graphic Displays of Basic Statistical Descriptions

  • Boxplots: Graphically show the five-number summary (minimum, Q1, median, Q3, maximum).

  • Histograms: Display data frequencies by creating bins of values along the X-axis.

  • Quantile plots: Pairs a value with its rank to show data distribution.

  • Scatter plots: Plot pairs of values of two attributes.

Data Quality Issues

  • The discussed measures of data quality include accuracy, completeness, consistency and believability, and interpretability

Data Cleaning

  • Incomplete data: Missing attribute values
  • Noisy data: Errors or variations
  • Inconsistency data: Different representations or values for the same entity
  • Methods for handling missing and noisy data include binning, smoothing, clustering, and detecting outliers
  • Handling: Ignore, imputation (fill in missing values; various techniques like mean, most frequent, global constant, etc.).

Data Integration

  • Combining data from multiple sources (databases, data cubes, files).
  • Goals include reducing noise, obtaining a more complete data view, and improving the speed and quality of data mining.
  • Methods include schema integration (e.g., merging metadata from different sources), entity identification (identifying real-world entities in different data sources having different names), and handling data value conflicts (e.g., conflicting values of the same attribute in different files for the same real-world object)

Data Transformation

  • Normalization: Scaling data to fall in a specified range (e.g., min-max, z-score, normalization by decimal scaling)
  • Discretization: Converting continuous data into categorical data (e.g., equal-width, equal-frequency)
  • Data compression : Reduce data size.
  • Sampling: Reducing data set size by selecting a representative subset.

Automatic Concept Hierarchy Generation

  • Methods to automatically create a hierarchical structure from attributes,
  • Examples (weekday, month, quarter, year, country, province/state, city, street) are included.

Sampling

  • Methods for selecting a representative subset from a large data set.
  • Sampling methods include simple random sampling, stratified sampling.

Cosine Similarity

  • Cosine similarity is a measure of the similarity between two vectors (e.g., term-frequency vectors, gene feature vectors).
  • It's used to determine the degree of similarity between two documents (or other objects) based on the angle between their vectors in a vector space.
  • The formula for cosine similarity is given.

Studying That Suits You

Use AI to generate personalized quizzes and flashcards to suit your learning preferences.

Quiz Team

Related Documents

Description

Test your knowledge on data attributes including nominal, ordinal, interval, and ratio scales. Learn about handling missing values, data noise, and the purpose of binning in data processing. This quiz covers key concepts important for data handling and analysis.

More Like This

Data Visualization Principles
219 questions

Data Visualization Principles

AffluentRisingAction9914 avatar
AffluentRisingAction9914
Data Types and Categorization Chapter 4
40 questions
Use Quizgecko on...
Browser
Browser