PR 2 Reviewer

Research Instrument Validity and Reliability

• Research instruments are tools used to collect data for specific research problems. Common ones include performance tests, questionnaires, interviews, and observation checklists.
• Quantitative research often uses questionnaires and performance tests; qualitative research commonly utilizes interviews and observation checklists. However, observation checklists and interviews can be used in quantitative research, too, if the data is translated into numerical form.

Characteristics of a Good Research Instrument

• Concise: Research instruments should be brief enough to encourage thorough responses without being overly tedious.
• Sequential: Questions should progress in a logical, structured manner, typically from simple to more complex.
• Valid and Reliable: Instruments should accurately measure what they intend to and provide consistent results.
• Easily Tabulated: Instruments need to be designed so that data collection can be easily organized for analysis.

Developing Research Instruments

• Adopt an existing instrument: Utilize instruments previously developed for similar studies.
• Modify an existing instrument: Adapt an existing one to fit the specific needs of the current research.
• Create a new instrument: Design a new instrument based on the research question and variables.

Common Scales

• Likert Scale: A commonly used scale in quantitative research. It prompts respondents to rate statements on a scale (e.g., strongly agree to strongly disagree).
• Semantic Differential: This scale uses bipolar adjectives to assess respondents' perceptions.

Types of Validity

• Face Validity: A foundational assessment; does the instrument appear to measure what it intends.
• Content Validity: Does the instrument assess all relevant aspects of the concept being measured? Expert input is important.
• Construct Validity: The instrument's measurements align with the theoretical concept.
• Concurrent Validity: A new instrument results in similar results to existing, validated measures.
• Predictive Validity: Instruments accurately predict future performance or behaviors.

Types of Reliability

• Test-retest Reliability: Consistency of results across repeated administrations of the instrument.
• Equivalent Forms Reliability: Consistency between different forms of the same instrument, (e.g., two different versions of a questionnaire).
• Internal Consistency Reliability: The items on an instrument measure the same construct (e.g., Cronbach's alpha).

Data Collection and Analysis

• Before: Prepare instruments, identify participants, determine sample size, obtain necessary permits/permissions, and pilot-test tools.
• During: Administer instruments or interventions, record responses to ensure data collection is accurately and ethically conducted.
• After: Summarize data, analyze data using appropriate techniques to answer the research question, and account for gaps in data collection.

Types of Data Analysis

• Descriptive Statistics: Summarize data (e.g., frequency distribution, averages).
• Inferential Statistics: Draw conclusions from the data (e.g., testing hypotheses).

Specific Statistical Techniques for Different Variables

• Univariate Analysis: Analyze a single variable.
• Bivariate Analysis: Investigate the relationship between two variables.
• Multivariate Analysis: Explore the relationship among multiple variables.

Test of Difference Between Two Data Sets

• Paired t-tests: Measuring the difference on a single variable between two measurements from the same person or group (e.g. before and after).
• Independent samples t-tests: Measuring the difference in a single variable between two distinct groups.
• Mann-Whitney U Test: Non-parametric test for interval/ordinal, independent. Wilcoxon Signed-Rank Test: Non-parametric; ordinal, 2 related groups; measures difference in a single variable between two related measurements from the same person/group.

Test of Difference between 2 or More Data Sets

• ANOVA: Determine if there are statistically significant differences in means among 3+ groups on an interval measure.

Types of Questions

• Recall Questions: Respondents retrieve information from memory.
• Recognition Questions: Respondents select answers from a list of options.
• Open-ended Questions: Respondents freely express their thoughts.

Data Collection

• Questionnaires: A series of questions used to gather data from participants.
• Interviews: Structured conversations used to collect in-depth information.
• Observation: Watching and recording behaviors or events.

Types of Tests

• Standardized tests: Tests with established norms and scoring that can be administered on large groups.
• Non-standardized tests: Tests without established norms; may differ slightly in scores depending on the person giving the test.