Big Data in Social Sciences & Reproducible Research

Study Notes

The world is surrounded by data.
Social media data, GIS data, economic data, military data, and data from randomized experiments/surveys are new data sources that have increased significantly in recent years.
New substantive ideas and data analysis tools are required to work with these new data sources.
The shift to new data has led to everyone analyzing data, not just statisticians, due to advancements like the internet and computing revolution.
Quantitative reasoning is essential for analyzing, interpreting, describing, and evaluating data to make good decisions in society and at work.

Comments should explain why the code does something, not how or what.
Comments should be updated when the code changes.
Object names must start with a letter and can only contain letters, numbers, underscores, and spaces.
Different naming conventions exist: snake_case (recommended by the professor), camelCase, PascalCase, and kebab-case.
Place spaces on either side of mathematical operators (except for ^) to improve code readability and conciseness.
Section code with comments (e.g., "#load data", "#plot data") as the script grows longer.

Reproducible research can be exactly redone given the materials used.
Another researcher should be able to reproduce results with the same code, data, and environment.
Code, dataset, and environment must be released.
Document the workflow to answer questions about the original dataset, data transformations, analysis done, and how the paper was built.

An R project bundles work in a portable, self-contained folder containing all relevant data and code.
setwd() sets the working directory, which is the folder R reads and saves files from by default.
A project is a working directory designated with a .RProj file.
When opening a project, the working directory automatically sets to the directory containing the .RProj file.

Quarto integrates code and natural language in "literate programming."
It is the successor of R Markdown, which allowed including R code chunks.
Quarto is a markup language similar to HTML or LaTeX.
It creates live documents where code executes and forms part of the document.
It allows compilation into HTML, PDFs, but this can take time as the code needs to run.