R Data Types and Data.Table Subsetting

Study Notes

Data.table subsetting structure: DT[i, j, by], where DT is the data.table, i is the rows, j is the columns, and by is the grouping variable.

Logical: used for logical data (TRUE or FALSE).
Integer: used for whole numbers (e.g., 0, 1, 2).
Numeric: used for real numbers (e.g., 1.1, 4.8) and can also be used for integer data, but is less efficient.
Factor: a special representation of numeric data for discrete data (e.g., study condition coded as 0, 1, 2).
Characters: used for text type data (e.g., names, qualitative data).

Logical operators: return TRUE or FALSE and are used for data management, such as finding outliers, checking values, and recoding continuous variables.
Examples of operators:
- = or %ge%: Greater than or equal
- %gl%: Greater than AND less than
- %gel%: Greater than or equal AND less than
- %gle%: Greater than AND less than or equal
- %gele%: Greater than or equal AND less than or equal
- %in%: In
- %!in% or %nin%: Not in
- %c%: Chain operations on the RHS together
- %e%: Set operator, to use set notation

Subsetting data: a common task in analyses, often used to exclude outliers or select specific participants.
The order of subsetting matters, and it's essential to consider the impact of different orders.

Rules for data merges in R:
- One join at a time
- The x dataset is always on the left, and the y dataset is always on the right
Types of joins:
- Natural join: resulting data has only rows present in both x and y (all = FALSE)
- Full outer join: resulting data has all rows in x and all rows in y (all = TRUE)
- Left outer join: resulting data has all rows in x (all.x = TRUE)
- Right outer join: resulting data has all rows in y (all.y = TRUE)

Necessary for repeated measures/longitudinal/panel data
Two types of data structures:
- Wide: each measure has a separately-named variable for each time point it was measured
- Long: time point (or wave) is a variable, and IDs will have multiple rows
Characteristics of wide data:
- Easy to read and interpret
- Each individual entity occupies their own row, and each of their variables occupies a single column
- Generally considered people-friendly
Characteristics of long data:
- Machine-friendly data structure
- Easier to perform functions like filtering, aggregating, and transforming
- Often used in data analysis and modeling