Statistics and Data Analysis Basics

Rejecting the null hypothesis (H₀) indicates that there is enough statistical evidence to support the alternative hypothesis (H₁), implying that the observed differences in data are unlikely due to random chance alone.

A bar graph is suitable for displaying data where the independent variable is categorical, meaning it has distinct, unordered categories like months, age groups, or animal species.

A bar graph is the best way to show discrete group data, like months of the year, age groups, shoe sizes, and animals, as it clearly displays the frequency or count of each category.

This code filters rows within each defined 'category' and keeps only the row with the minimum 'rank' value in that group.

Data interpretation involves analyzing collected data to derive insights and conclusions, drawing meaning and understanding from the patterns and trends observed.

The initial step in cleaning duplicate rows is sorting the dataset by a key variable and removing rows that are identical across all columns, which is often a more effective approach.

Diagnostic analysis involves identifying the underlying causes of a problem, in this case, sales decline. Techniques like cohort analysis or time series analysis are commonly employed to uncover the reasons for the decline.

If the p-value is less than the significance level (α), you can reject the null hypothesis, as the observed data is unlikely to have occurred under the null hypothesis.

Data validation is essential to ensure data accuracy and quality before analysis to ensure that the data is reliable and usable for drawing meaningful insights.

The summarize() function in R is used to calculate summary statistics for a dataset, such as the sum of a column. Here, it calculates the sum of the 'sales' column and creates a new column named 'total_sales'.

Exploratory data analysis (EDA) techniques like correlation analysis and hypothesis testing are often used to identify relationships and uncover potential causes in diagnostic analysis.

The dbGetQuery() function from the DBI package in R is used to execute SQL queries and retrieve the resulting data into R as a data frame.

The pipe operator (%>%) in dplyr allows chaining multiple functions together in a sequential manner, enabling cleaner and more readable code for data transformation and analysis.

Prescriptive analysis recommends specific actions or strategies based on data-driven insights to improve outcomes. In a retail context, this involves providing personalized marketing recommendations based on customer preferences and behavior.

The Area Under the Receiver Operating Characteristic Curve (AUC-ROC) is a relevant metric for evaluating classification models, as it measures the overall quality of a model's predictions across different classification thresholds.

High variance in data indicates that data points are widely spread out, meaning there is a greater variation among the data points.

The stringsAsFactors = FALSE argument in the read.csv() function prevents the automatic conversion of strings into factors during data import, preserving string data types.

The na.omit() function in R is designed to remove rows containing missing values (NA) from a data frame.

Velocity, the speed at which data is generated and processed, necessitates event-driven architectures that can process streams of data in real-time. This aligns with near-real-time analytics requirements.

A variable measured on an ordinal scale has categories that can be ranked and ordered, but the differences between the categories are not necessarily equal. For example, customer satisfaction on a 1-5 scale.

A scenario that integrates both predictive and prescriptive analysis involves forecasting demand for a new product and then using the prediction to recommend optimal inventory levels, guiding decision-making based on future demand.

The read_excel() function in R is used for reading data from Excel files, but unlike the other statements, it doesn't have the capability to read a password-protected Excel file.

Descriptive analytics primarily focuses on summarizing and describing historical data, providing a clear overview of past trends and patterns.

The chi-square test of independence is used to examine the association between two categorical variables in a dataset. If the result is statistically significant, it indicates that there is a relationship between the variables.

Grouping rare categories together into an "Other" category can simplify high-cardinality categorical variables by reducing the number of distinct categories.

The scan() function is more flexible for reading data, allowing different data structures like vectors, lists, or matrices. While read.table is optimized for creating data frames, scan offers more versatility.

Oversampling the minority class or undersampling the majority class is a technique for addressing imbalanced classes in a dataset, to ensure fairer representation of minority and majority groups.

Variety in big data refers to the diverse types of data collected from various sources, including structured and unstructured data. It presents challenges for integrating and managing data from different formats.

This code efficiently filters the top 10 rows based on the sales column, using slice_max to directly select the highest values from the sorted data.

A key difference between tibbles and traditional data frames is that tibbles do not support row names. They are more concise and better for interacting with other packages.

Handling outliers due to data entry errors is typically best addressed by correcting or removing the erroneous data points. It doesn't necessarily require complex transformations.

Cloud-based platforms provide scalability and flexible resource allocation, dynamically adapting to varying data loads without constant infrastructure adjustments. This is crucial for big data's Volume challenge.

The str(df) function in R provides a concise overview of the data frame's structure, including the data types of each column, the number of rows and columns, and the first few rows of the data.

A statistical hypothesis is a testable statement regarding the population parameters, for example, a hypothesis about the mean cholesterol levels of two groups with different diets.

A Type I error occurs when the null hypothesis is incorrectly rejected when it is actually true. This implies concluding that there is an effect when there is none.

Data normalization involves standardizing data values to a common scale. It helps align the data to avoid dominance of certain variables with larger scales during analysis.

Data wrangling encompasses the steps taken to prepare and transform raw data into a format that is suitable for analysis. This involves cleaning, tidying, and reshaping the data.

Histograms, boxplots, and scatterplots can all be used to identify outliers in a dataset. Histograms show the distribution, boxplots highlight the IQR, and scatterplots reveal potential unusual values.

The read_csv() function from the readr package is used to import CSV files and returns the output as a tibble in R.