Combined Midterm Questions - Research Design Fall 2024 PDF

**1) Which of the following is an example of a dependent variable in an experiment?**\ a) The type of teaching method used in a classroom\ b) The number of hours a student studies for an exam\ c) The test scores of students after being taught using different methods **(Correct)**\ d) The school subj...

**1) Which of the following is an example of a dependent variable in an experiment?**\ a) The type of teaching method used in a classroom\ b) The number of hours a student studies for an exam\ c) The test scores of students after being taught using different methods **(Correct)**\ d) The school subject being taught **2) Which of the following best describes a discrete variable?**\ a) A variable that falls along a continuum and is not limited to a certain number of values\ b) A variable that assumes a range of numerical values\ c) A variable that falls into separate categories with no intermediate values **(Correct)**\ d) A variable that can change or take on different characteristics over time **3) Which of the following is a threat to internal validity in research designs?**\ a) Interaction of selection and treatment\ b) Statistical power\ c) History events occurring between the pretest and posttest **(Correct)**\ d) Generalizability to other settings **4) Which type of research validity refers to how well a study can generalize its findings to other settings, populations, or times?**\ a) Internal validity\ b) External validity **(Correct)**\ c) Statistical conclusion validity\ d) Construct validity **5) What are the defining characteristics of experimental research designs?**\ a) Manipulation of variables and random assignment to groups **(Correct)**\ b) Observational data collection and control groups\ c) Archival data analysis and statistical control\ d) Random sampling and use of natural settings **6) Which of the following is an advantage of using a pretest in an experimental design?**\ a) It reduces the number of participants needed\ b) It ensures groups are equivalent before treatment **(Correct)**\ c) It eliminates the need for random assignment\ d) It allows for observational data collection **7) Which of the following is a method to control for order and sequencing effects in an experiment?**\ a) Matching\ b) Block randomization **(Correct)**\ c) Using archival data\ d) Increasing sample size **8) What is the primary difference between true experimental designs and quasi-experimental designs?**\ a) Use of control groups\ b) Random assignment of participants **(Correct)**\ c) Manipulation of independent variables\ d) Use of observational data **9) Which of the following is an example of a quasi-experimental design?**\ a) A study comparing employee productivity between two companies, where employees are randomly assigned to different work conditions\ b) A study comparing the effect of a new teaching method on two classrooms, where the classes were already formed and not randomly assigned **(Correct)**\ c) A clinical trial where participants are randomly assigned to receive either a new drug or a placebo\ d) An experiment testing the effect of a marketing campaign on customer behavior with participants randomly selected from the population **10) Which of the following is a key characteristic of naturalistic observation in research?**\ a)\ b) The researcher participates in the group\ c) The researcher interacts with participants\ d) The researcher observes behavior without disturbing it **(Correct)** 1. A critic argues that the inferred variation in construct Y due to construct X, is due to construct Z instead. What type of validity is threatened? a. Construct validity b. Statistical conclusion validity c. Internal validity d. External validity e. Construct-related validity 2. A researcher wants to increase the statistical ability of his analysis to detect an effect where it exists, which of the following should he do? a. b. c. Meet test assumptions d. Increase sample size e. Use a different sample 3. One of the methods for assessing reliability is: a. Pretest-posttest b. Test -- Retest c. Pretest-test-posttest d. Test -- Posttest e. 4. Which of the following is a type of construct-related validity? a. Face validity b. Convergent validity c. External validity d. Bivariate validity e. Internal validity 5. A researcher concludes that the newly formulated speed inducement therapy improves walking ability of young children. A pretest was carried out for 7 month olds and a posttest carried out at 9 months. The 9 month result shows a significant walking improvement in the children. What type of threat to internal validity may be involved? a. History b. Maturation c. Testing d. Selection e. Regression 6. Dr. Edwards advice against using convenient samples because it lacks \-\-\-\-\-\-\-\-\-\-\-\-- a. Rigor b. Accuracy c. Representativeness d. Timeliness e. Sufficiency 7. I am considering emotional intelligence as a label for observed communication skills at workplace. Dr. Edwards thinks the label lacks relevance and suggests that I consider another label. Dr. Edwards' rationale is because he is concerned about: a. construct validity b. internal validity c. external validity d. statistical conclusion validity e. 8. Validity refers to the approximate truth about \-\-\-\-\-\-\-\-- a. A sample b. A design c. A statistic d. A measure e. An inference 9. According to Highhouse (2009), experimental designs have one common criticism. Which is it? a. Equivalence of groups b. Setting a baseline. c. Effects of testing d. Ability to generalize e. Small sample size 10. Which is a primary advantage of within-person design? a. Not many participants needed b. Equivalence is not certain c. Effects of repeated testing d. Dependability of treatment effects e. Irreversibility of treatment **1) Which of the following is NOT one of the main processes (objectives) of science as described in the Notes ?** A\) Description B\) Explanation C\) Intuition D\) Prediction **Correct Answer:** C) Intuition **2) Which of the following best describes the single most important element of the scientific process?** A\) Replication B\) Operational definition C\) Control D\) Theoretical exploration **Correct Answer:** C) Control **3) Which of the following measurement scales is characterized by having equal intervals between values but lacks a true zero point?** A\) Nominal Scale\ B) Ordinal Scale\ C) Interval Scale\ D) Ratio Scale **Correct Answer:** C) Interval Scale **4) Which type of validity refers to the effectiveness of a test in predicting an individual\'s behavior in specific situations?** A\) Content--related validity\ B) Construct--related validity\ C) Face validity\ D) Criterion--related validity **Correct Answer:** D) Criterion--related validity **5) Which type of validity assesses whether the results of a study can be generalized beyond the specific conditions of the experiment?** A\) Internal Validity\ B) External Validity\ C) Statistical Conclusion Validity\ D) Construct Validity **Correct Answer:** B) External Validity **6) Which type of validity focuses on the extent to which a study accurately measures the relationship between variables within the experimental context?** A\) Internal Validity\ B) External Validity\ C) Statistical Conclusion Validity\ D) Construct Validity **Correct Answer:** A) Internal Validity **7) What does the power of a statistical test refer to?** A\) The probability of rejecting the null hypothesis when it is true.\ B) The probability of correctly rejecting the null hypothesis when it is false.\ C) The likelihood of obtaining a significant result purely by chance.\ D) The ability of a test to accurately estimate the population parameter. **Correct Answer:** B) The probability of correctly rejecting the null hypothesis when it is false. **8) Which of** **the following is NOT a reason for** **conducting a pretest in research?** A\) Equivalence of groups\ B) Establishing a baseline\ C) Measuring the overall outcome of the study\ D) Assessing effects of testing or practice effects **Correct Answer:** C) Measuring the overall outcome of the study **9) What does the condition of contiguity refer to in the context of inferring causality?** A\) The cause must be tested in isolation from the effect.\ B) The effect must occur at the same time as the cause.\ C) The presumed cause and effect must occur close together in time and space.\ D) The cause must always produce the same effect under all conditions. **Correct Answer:** C) The presumed cause and effect must occur close together in time and space. **10) Which of the following statements accurately reflects a caveat to determining causality?** A\) Cause--effect relationships can always be established through experimental methods.\ B) A significant correlation between two variables definitively proves a causal relationship.\ C) It is impossible to claim that cause--effect relationships are true; we can only state they have not been falsified.\ D) Correlational methods are reliable tools for establishing causal relationships. **Correct Answer:** C) It is impossible to claim that cause--effect relationships are true; we can only state they have not been falsified. **11) Which of the following** **actions is NOT typically** **associated with achieving statistical control in a study?** A\) First determining if the design permits analysis by accepted statistical methods.\ B) Increasing the number of trials to gather more data points.\ C) Using a specialized statistical technique to enhance control, such as ANCOVA or partial correlations.\ D) Increasing the statistical power of a design, for example, by increasing sample size. **Correct Answer:** B) Increasing the number of trials to gather more data points. **12) Which statement accurately differentiates between moderation and mediation in psychological research?** A\) Moderation examines the process through which one variable affects another, while mediation tests if the relationship between two variables changes depending on a third variable.\ B) Mediation refers to the influence of a third variable on the relationship between two other variables, while moderation assesses how a third variable affects the strength or direction of that relationship.\ C) Both moderation and mediation involve the same statistical techniques and have identical interpretations.\ D) Mediation focuses on correlation, while moderation is concerned with causation. **Correct Answer:** B) Mediation refers to the influence of a third variable on the relationship between two other variables, while moderation assesses how a third variable affects the strength or direction of that relationship. **Items for mid** 1\. What is the main effect in research? A\) The overall impact of one independent variable B\) The interaction between two variables C\) The error in measurement D\) The sample size Answer: A) The overall impact of one independent variable 2\. What is an advantage of within--subjects design? A\) Equivalence is certain B\) Effects of repeated testing C\) Dependability of treatment effects D\) Irreversibility of treatment Answer: A) Equivalence is certain 3\. What is an advantage of between--subjects design? A\) Equivalence is certain B\) Effects of testing are minimized C\) Equivalency is less assured D\) Greater number of participants needed Answer: B) Effects of testing are minimized 4\. What does \'Realism\' assume in the scientific method? A\) Objects perceived have an existence outside the mind B\) Reasoning and logic are the only way to solve problems C\) Phenomena exist in recurring patterns D\) Events happen because of preceding causes Answer: A) Objects perceived have an existence outside the mind 5\. What is one limitation of using common sense as a method of acquiring knowledge? A\) It can generate new knowledge B\) It is theoretical in nature C\) It is pragmatic but does not explain the \"why\" behind something D\) It remains unchanged over time and across cultures Answer: C) It is pragmatic but does not explain the \"why\" behind something 6\. What is an independent variable in a research study? A\) The condition manipulated to determine its effect on behavior B\) The condition measured to reflect the effect of another variable C\) A constant that does not change D\) An attribute of a phenomenon that cannot be measured Answer: A) The condition manipulated to determine its effect on behavior 7\. What does \'construct validity\' refer to? A\) The ability to generalize the results to other populations B\) The extent to which labels are relevant to the theory being studied C\) The reliability of the measurement tool D\) The ability to eliminate alternative explanations Answer: B) The extent to which labels are relevant to the theory being studied 8\. Which of the following is a method for assessing test reliability? A\) Test-retest reliability B\) Statistical conclusion validity C\) Criterion-related validity D\) Construct-related validity Answer: A) Test-retest reliability 9\. Which research design involves manipulating variables and using random assignment? A\) Correlational design B\) Experimental design C\) Quasi-experimental design D\) Observational design Answer: B) Experimental design 10\. Which type of validity is concerned with whether the IV and DV are statistically related? A\) Internal validity B\) External validity C\) Construct validity D\) Statistical conclusion validity Answer: D) Statistical conclusion validity **Midterm Questions** 1. **Control**, the most important element of the scientific process, refers to which of the following in a given explanation for a phenomenon? a. The ability to remove or account for alternative explanations. b. The strength of one variable's effect on another variable. c. The ability of a statistical test to identify present relationships. 2. A **Quantitative Variable** always: d. Is continuous. e. Is distinct from a qualitative variable. f. Varies in amount. 3. A **Ratio scale**: g. Lacks a known interval between points. h. Is common in behavioral/social sciences i. Has a known true zero-point. 4. **Content-related validity** involves: j. Empirically and objectively determining a test's quality. k. Expert judgment of a test's purported and real measurements. l. How well a test predicts an individual's future behavior. 5. **Internal validity** is the extent to which: m. Findings can be generalized to other scenarios. n. Causal inference can be ascertained. o. Statistical inferences from data are valid. 6. Which of the following is a threat to **External validity?** p. Low ecological validity. q. Attrition during the study. r. The Hawthorne Effect. 7. Which of the following is true regarding **Construct validity**? s. Internal validity is first needed to obtain it. t. It deals with measures and statistical tests. u. It increases by using random sampling. 8. **Experimental research design** necessarily requires the ability to manipulate variables and: v. A lab setting. w. A within-subjects control group. x. Random assignment. 9. **Quasi-experimental designs** allow the researcher to y. Most effectively derive causal inference. z. Research without random assignment. a. Exercise greater control over variables. 10. Of the following, which is an experimental research design to avoid: b. One group with a pretest, treatment, and posttest. c. Two groups with a pretest, control group, treatment, and posttest. d. One group with a pretest, treatment, within-subject control condition, and posttest. **Answer Key** 1. A 2. C 3. C 4. B 5. B 6. A 7. A 8. C 9. B 10. A Good questions but I think all of these are better as short answer and not MC. **Midterm questions - Research Methods** Q1: What types of dependent and independent variables exist? And how are they classified? A: There are continuous (take an undetermined range of values) and discrete variables (falls into separate categories, such as male/female). We can classify these variables as quantitative (varies in amount) or qualitative (varies in kind, for example, sex). Q2: What are the different levels of measurement, which one is less commonly used in organization science and why? A. labels, ordinal scale, nominal scale, ratio scale and interval scale. The less commonly used measurement is the interval scale, as it represents the distance from a zero-point. B. labels, ordinal scale, nominal scale, and interval scale. The less commonly used measurement is the interval scale, as it represents the distance from a zero-point and between persons or objects. C. labels, ordinal scale, nominal scale, ratio scale and interval scale. The less commonly used measurement is the ratio scale, as it represents the distance from a zero-point and between persons or objects. D. labels, ordinal scale, nominal scale, ratio scale and interval scale. The less commonly used measurement is the ratio scale, as it represents how people or objects are ordered with respect to an attribute. A: C) Q3: What are the types of test and measurement validity and which one would be the most appropriate for your paper idea for the semester, and why? A: The types of test validity are criterion-related, content-related, and construct-related. The chosen type of test validity will be different for each student. Q4: List the different research validities and their corresponding threats: A: - Internal validity: the extent to which we can infer the causal relationship between a DV and IV. Its threats are history (the effects may happen because of an external event that happened between the pretest and posttest), maturation (source of error given the time between measurements, for different periods research), testing (due to the number of times responses are measured), mortality/attrition, selection (samples must represent the population), regression effects (tendency of higher scores to then score closer to the mean). - External validity: the causal relationships can be generalized to other people, settings, times. Threats are: other participants, settings and times. - Statistical conclusion validity: IV and DV are statistically related. Threats: low statistical power, violations of statistical tests' assumptions, low reliability of measures. - Construct validity: research results should support the underlying theory chosen (labels used should be relevant to theory studied). Threats: lost connection between theory and the experiment, evaluation apprehension, experimenter expectancies. Q5: What is random assignment and what are some of the threats regarding this design? A. The researchers choose a representative group of the population, where each member of the population has an equal and independent chance of being part of the sample. Some threats are the possible manipulation or a treatment imposed in the sample group. B. The researchers choose a representative group of the population, where each member of the population has an equal and independent chance of being part of the sample. Some threats are the possibility of manipulation or imposed treatment as well as not knowing if the causal relationship between IV and DV are by chance of the actual effect. C. Having groups of participants where every member of the sample has the same chance of being assigned to either group. Some threats are the possibility of manipulation or imposed treatment as well as not knowing if the causal relationship between IV and DV are by chance of the actual effect. A: C) Q6: How is the Solomon Four-Group Design arranged and what are the comparisons analyzed with it? A: We have four groups for our total sample. Group 1 will have a pretest, treatment and posttest. Group 2 will have only a treatment and posttest. Group 3 will have only a pretest and posttest and group 4 will have only a posttest. The objective with this arrangement is to being able to study and compare the effect of having a treatment or no treatment at all, the effect of having a pretest on the treatment, and the effect of having a pretest or no pretest at all. This gives us a better understanding of the causal relationship between the IV (with the treatment) and the DV. Q7: What are the differences between the within and between subjects design and which one is better to use? A: Within subjects design allows each participant to experience all the conditions of the experiment while in between subjects design each participant will experience only one condition of the experiment. The choice of which one to use depends on the study purpose and design, but for the between subjects design, more participants are needed, while for within subjects design, despite needing a smaller sample, there's the risk of having effects by repeated testing or not being able to reverse the treatment (as all participants experience all experiment conditions). A. B. C. A. B. C. Q10: What are the strategies to achieve control in experimental design? A. Matching (match participants by some characteristics), building nuisance variables, instrumentation of response (use an objective and reliable measurement or instrument). B. Random assignment (match participants by some characteristics), Matching (rules out alternative interpretations of groups by chance), statistical control (use specialized statistical techniques, increase number of trials and statistical power), building nuisance variables, subject as own control (participants will experience every condition of the experiment, although that can generate issues with order and sequence effects, for example), instrumentation of response (use an objective and reliable measurement or instrument). C. Random assignment (rules out alternative interpretations of groups by chance), Matching (match participants by some characteristics), statistical control (use specialized statistical techniques, increase number of trials and statistical power), building nuisance variables, subject as own control (participants will experience every condition of the experiment, although that can generate issues with order and sequence effects, for example), instrumentation of response (use an objective and reliable measurement or instrument). Answer: C 1. Which of the following is a characteristic of the scientific approach? a\) Empirical b\) Subjective c\) Dogmatic d\) Authoritative **Answer: a) Empirical** 2. Which of the following is an example of a continuous variable? a\) Height b\) Gender c\) College Major d\) Political Affiliation **Answer: a) Height** 3. Which type of validity is concerned with the extent to which a cause-effect relationship exists between the independent and dependent variables? a\) Internal validity b\) External validity c\) Content validity d\) Construct validity **Answer: a) Internal validity** 4. Which of the following is a threat to external validity? a\) Interaction of selection and treatment b\) History c\) Maturation d\) Testing **Answer: a) Interaction of selection and treatment** 5. Which of the following is a defining characteristic of experimental designs? a\) Manipulation of variables b\) Non-random assignment to groups c\) Use of correlational methods d\) Lack of a control group **Answer: a) Manipulation of variables** 6. What is a potential reason to use a pretest in an experimental design? a\) To mask the effects of the independent variable b\) To establish the equivalence of groups c\) To increase the likelihood of attrition d\) To introduce confounding variables **Answer: b) To establish the equivalence of groups** 7. Which control technique involves having each participant experience every condition of the experiment? a\) Subject as own control b\) Random assignment to groups c\) Matching d\) Statistical control **Answer: a) Subject as own control** 8. Which research design involves the manipulation of an independent variable but lacks random assignment to groups? a\) True experimental design b\) Correlational design c\) Quasi-experimental design d\) Observational design **Answer: c) Quasi-experimental design** 9. Which type of quasi-experimental design involves comparing a group\'s trend of data before and after a treatment? a\) Interrupted time-series design b\) Nonequivalent control group design c\) Repeated treatment design d\) Mixed factorial design **Answer: a) Interrupted time-series design** 10. What is a defining characteristic of observational designs? a\) The presence of manipulation b\) The ability to make causal inferences c\) The use of questionnaires d\) The manner in which data is collected **Answer: d) The manner in which data is collected** 11. Which research design focuses on measuring the degree of relationship between two or more variables without manipulation? a\) True experimental design b\) Quasi-experimental design c\) Observational design d\) Correlational design **Answer: d) Correlational design** 12. Which type of survey question provides respondents with a limited set of alternatives to choose from? a\) Closed-ended b\) Open-ended c\) Neither a nor b d\) Cannot be determined **Answer: a) Closed-ended** 13. What is social desirability in survey research? a\) The tendency to present oneself in a socially desirable manner b\) The tendency to present oneself in a socially undesirable manner c\) The tendency to avoid answering sensitive questions d\) The tendency to provide neutral responses **Answer: a) The tendency to present oneself in a socially desirable manner** 14. Which research design involves studying the same cohort or group of individuals over a long period of time? a\) Cross-sectional b\) Longitudinal c\) Neither a nor b d\) Cannot be determined **Answer: b) Longitudinal** 15. Which specific threat to internal validity is exclusive to longitudinal designs? a\) History or cohort effects b\) Selective dropout c\) Practice effects d\) Selective survival **Answer: b) Selective dropout** 16. Which statistical method is used to quantitatively aggregate the results of several primary studies? a\) ANOVA b\) Correlation c\) Meta-analysis d\) Regression **Answer: c) Meta-analysis** General comment from Bryan: It seems as if most of these items are intended as short answer and that is fine. I just wanted all of the students to know that many of these will not be MC questions. But, the content is important to study. Also -- there will \*not\* be any questions on the test with a negation (a "not") in the stem. I quit editing after the first page. 1. What is the method not used for acquiring knowledge? a. Intuition b. Common Sense c. Science d. Authority 2. Which one of the following is not a working assumption of science? a. Realism b. Mysticism c. Rationality 3. List the role of theories a. Organizing knowledge and explaining laws. b. Predicting new laws c. Guiding research 4. Which one is not an element of the scientific process? a. Control b. Operational definition c. construct 5. Which one is the single most important element of the scientific process? a. Control b. Operational definition c. Replication **[VARIABLES AND MEASUREMENT]** 6. List some definitions of variables a. Symbol that can assume a range of numerical values. b. Some property of an organism/even that has been measured c. An aspect of a testing condition that can change or take on different characteristics with different conditions. d. An attribute of a phenomenon. 7. What are the different types of variables? a. Independent and dependent variables b. Continuous and discrete variables c. Quantitative & Qualitative variables 8. Fill in the blank [Independent Variable] is the condition manipulated as selected by the researcher to determine it effect on behavior. 9. Fill in the blank [Dependent Variable] is a measure of the behavior of the participant that reflects the effect of the independent variable. 10. Fill in the Blank [Continuous Variable] is one that falls along a continuum and is not limited to a certain number of values (distance or time). 11. Fill in the blank 12. Which one is not one of the commonly used methods to generate artificially discrete variables? a. Mean Split b. Median Split c. Mode Split d. Extreme group 13. Fill in the blank 14. Fill in the blank 15. List the five basic types of scales a. Label b. Nominal scale c. Ordinal scale d. Interval scale e. Ratio scale 16. Which scale is extremely rare in the behavioral and organization sciences: a. Nominal b. Ratio c. Interval 17. What are the two ways to evaluate measurement methods and instruments? a. Reliability b. validity 18. Fill in the blank 19. Fill in the blank [Validity] extent to which a method measures what it is supposed to measure. a. True b. False 20. Which one of the methods for assessing reliability is the most common? a. Test\--re-test reliability b. Alternate-forms reliability c. Split-half, odd-even d. Kuder-Richardson (KR20) reliability and coefficient alpha 21. Fill in the blank [Validity] measures the accuracy of a test 22. What are the three types of test validity? a. Criterion-related b. Content-related c. Construct-related 23. What is Criterion-related validity? The effectiveness of a test in predicting and individual's behavior in specific situation 24. What are the most common types of criterion-related validity? - Concurrent - Predictive - Postdictive 25. Define Construct related validity of a test? Answer: The extent to which the test may be said to measure a theoretical construct or trait. 26. What is a Construct? 27. What are the types of construct-related validity? a. Convergent validity b. Divergent or discriminant validity 28. Fill in the blank 29. Fill in the blank 30. What is Face Validity? It has to do with the extent to which a test or measure looks like it measures what it is supposed to **[RESEARCH VALIDITY]** 31. Fill in the blank 32. Which one is not a key criterion in evaluating validity? a. Concurrent b. Data/observation c. Drawing inferences 33. List the key criterion in evaluating any test, measure, or piece of research validity a. Data/observation b. Drawing inferences c. Appropriateness 34. Which is not a way how the concept of validity is used? a. Test and measure validity. b. Multi-trait and multi matrix method. c. Research or experiment validity 35. What are the items for test and measurement validity? a. Criterion-related b. Content-related c. Construct-related 36. Which is not an element of test and measurement validity? a. Criterion-related b. Internal-related c. Construct-related 37. Which one is not an element for Research or experimental validity a. Face validity b. Internal c. Statistical conclusion 38. Which is not a type of research validity? a. Internal validity b. Content-related validity c. Construct validity 39. List the commonly recognized types of research validity? a. Internal validity b. External validity c. Statistical conclusion validity d. Construct validity **(ACRONYMS: IESC)** 40. What is internal validity? Answer: the extent to which we can infer that a relationship between two variables is causal or that absence implies absence of cause. 41. What are the two types of internal validity? a. Extraneous variable. b. Confounding 42. Which one is not a threat to internal validity? a. History b. Population c. Maturation 43. List the threats to internal validity: - History - Maturation - Testing - Mortality/Attrition - Selection - Regression effects 44. What is external validity? Answer: The inference that presumed causal relationship can be generalized to and across alternate measure of cause and effect. 45. Which one is not a threat to external validity? a. Selection b. Ecological validity c. Temporal validity 46. List the types of threat to external validity a. Population validity b. Ecological Validity c. Temporal validity 47. What is field experiment? Answer: The best tradeoff between internal and external validity 48. Define Statistical Conclusion validity: Answer: Appropriateness of inference made from data as a result of conclusion from Statistical analysis. 49. Which is not a threat to Statistical validity? a. High Statistical power b. Reliability of measure c. Violated assumption of statistical test 50. What are the threats to Statistical conclusion validity? a. Low Statistical power b. Violated assumption of statistical test c. Reliability of measures 51. What is Construct validity? Answer: Related to construct-related validated and is a question of whether the research results support the theory underlying the research. 52. Which is a threat to construct validity? - Reliability of measures. - Evaluation apprehension. - Importance of trade-off. 53. What are the threats to construct validity? a. Loose connection between theory and experiment b. Evaluation apprehension c. Experimenter expectations - Sample size is used to increase statistical conclusion validity (T or F) 54. What is the rank order for validity? a. Statistical conclusion validity. b. Internal validity c. External validity **(ACRONYMS: SIE)** Research Methods Topic 3: Validity **Multiple Choice Questions** **1). Which of the following best describes when a conclusion based on research is considered valid?** A\) When the research can be replicated by other researchers.\ B) When the research findings correspond to the actual state of the world.\ C) When the research findings align with the researcher\'s initial hypothesis. ***B** is the correct answer because research is considered valid when its findings are accurate reflections of the real world, not just aligned with the hypothesis or replicable.\ * **2). Which of the following is NOT one of the commonly recognized types of research validity?** A\) Internal validity\ B) External validity\ C) Procedural validity *Internal validity refers to the extent to which the observed effects in a study can be attributed to the independent variable, and not to other extraneous factors. It is concerned with establishing a cause-and-effect relationship within the context of the study.* *External validity deals with the generalizability of the study\'s findings beyond the specific conditions or participants in the study. It asks whether the results can be applied to different populations, settings, and times. C is therefore the correct answer.* **3). What is a confounding variable in the context of internal validity?\ **A) A variable that systematically varies with the independent variable\ B) A variable that is manipulated by the researcher\ C) A variable that is completely unrelated to the dependent variable *A is correct because a confounding variable systematically varies with the IV and has the potential to influence the outcome, thereby threatening the internal validity of the study.* **4). What is internal validity primarily concerned with in research?\ **A) Whether the relationships observed are due to confounding variables.\ B) Whether the study\'s findings can be generalized to the real world.\ C) Whether the relationship between two variables is causal. *C is the correct answer because internal validity\'s primary concern is ensuring that any observed effects in the study are due to the independent variable and not other factors, thereby confirming a causal relationship between variables.* **5). Which of the following is an example of a confounding variable?\ **A) A study examining the effect of exercise (IV) on weight loss (DV), but diet is not controlled and could also influence weight loss.\ B) A study where the same test is administered to all participants.\ C) A study where participants are randomly assigned to groups to ensure fairness. *Someone who exercises regularly but eats poorly may not lose weight, whereas someone who eats a healthy diet but exercises less might still lose weight. This creates confusion, making it unclear whether weight loss is due to exercise or diet.* **6).** **What is external validity primarily concerned with?**\ A) Whether the study is free from extraneous variables.\ B) Whether the findings can be generalized to different participants, settings, and times.\ C) Whether the relationship between two variables is causal. *External validity is specifically about how well the study\'s findings can be generalized beyond the experimental conditions.* **7). Which of the following best describes statistical conclusion validity?** A\) The degree to which findings can be generalized to other settings or populations.\ B) The appropriateness of inferences made from data as a result of statistical analysis.\ C) The ability to establish a cause-and-effect relationship between variables. *Statistical conclusion validity is concerned with whether the conclusions or inferences made from the data are statistically sound and valid. Specifically, it focuses on whether the independent variable (IV) and dependent variable (DV) are statistically related based on the analysis.* **8). What is construct validity primarily concerned with?**\ A) The ability to generalize the study's findings to other populations.\ B) Whether the labels or concepts used in the study are theoretically relevant.\ C) The relationship between the independent and dependent variables. *Construct validity is about whether the terms and labels used in the study accurately reflect the theoretical concepts being investigated. It ensures that the constructs (e.g., emotional intelligence vs. communication skills) are the right ones for the study.* **9). Which of the following is a threat to construct validity?**\ A) Low statistical power\ B) Evaluation apprehension\ C) Generalizability of results *\ Evaluation apprehension refers to the tendency of research participants to alter their behavior because they know they are being studied, which can affect the construct validity of the study by influencing how constructs are measured.* **10). How can threats to construct validity, such as the "good-subject response" and evaluation apprehension, be minimized?**\ A) By using a larger sample size\ B) By applying double-blind or single-blind procedures\ C) By ensuring that the findings can be generalized to other settings *Double-blind and single-blind procedures help minimize biases such as experimenter expectations or participants acting in ways they believe the researcher wants, which can undermine construct validity.* **Short Answer Questions** **1). Define internal validity and explain how confounding variables can threaten it.** *Internal validity refers to the extent to which we can infer that a cause-and-effect relationship exists between the independent variable (IV) and the dependent variable (DV) in a study. It ensures that the observed effect is due to the manipulation of the IV and not influenced by external factors. Confounding variables threaten internal validity because they systematically vary with the IV and may influence the DV, making it difficult to determine whether the IV or the confounding variable caused the observed effect.* **2). What is external validity and what are some factors that can threaten it?** **\ ***External validity is the degree to which the findings of a study can be generalized to other populations, settings, or times beyond the specific conditions of the study. Factors that can threaten external validity include differences in participant characteristics (interaction of selection and treatment), variations in settings (interaction of setting and treatment), and changes in time (interaction of history and treatment). These threats limit the ability to apply the study's results to different situations or groups.* **3). Describe construct validity and provide an example of a threat to it.** **\ ***Construct validity refers to the extent to which the labels or concepts used in a study accurately represent the theoretical constructs being measured. It ensures that the research results support the theory underlying the study. An example of a threat to construct validity is evaluation apprehension, where participants alter their behavior because they are aware they are being studied, which can distort the accurate measurement of the theoretical construct. For instance, if participants behave more politely in a study on communication skills because they know they are being observed, it could affect the study's construct validity.* *\ * Midterm Questions \*Correct answers are bolded and highlighted\* Multiple Choice Questions 1. What is the primary purpose of operational definitions in research? a. **Define variables in measurable terms** b. Ensure the funding of research projects c. Eliminate limitations in sample size 2. Which of the following constructs presents a challenge in operational definition compared to others like age or gender? d. Number of accidents e. Firm performance f. **Charismatic leadership** 3. Which of the following is considered the single most important element of the scientific process? g. Operational definition h. **Control** i. Replication 4. What is one of the primary advantages of utilizing the scientific process? j. **It promotes objective observation independent of bias** k. It relies heavily on historical interpretations l. It seeks to confirm existing beliefs without challenging them 5. Which method of acquiring knowledge is characterized by spontaneous judgment not based on mental steps? m. Science n. Authority o. **Intuition** 6. Which belief supports the idea that phenomena follow the same laws at all times and places? p. **Regularity** q. Realism r. Causality 7. Which assumption of science proposes that it is possible to learn answers to posed questions, and that the only limitations are time and resources? s. Causality t. Realism u. **Discoverability** 8. Which of the following is a role of theories? v. **Guiding research** w. Seeking control x. 9. What does causality imply in the context of science? y. **All events occur due to preceding causes** z. Events are random and unpredictable a. Knowledge can only be gained through intuition 10. What is a limitation of using mysticism as a method of knowledge acquisition? b. It is based solely on logical reasoning c. **It involves personal experiences** d. It is grounded in empirical evidence Short Answer Questions 11. What are the two advantages of the scientific method? e. **The primary advantage of science is that it is based on objective observation (observation independent of opinion or bias). It also allows us to establish the superiority of one belief over the other** 12. What are the 4 processes (objectives of science)? 1. **Description** 2. **Explanation (development of theories)** 3. **Prediction (formulated from theories)** 4. **Control** **[Mid Term Questions]** **1. What is the primary concern of experimental control in research?** A. Ruling out threats to research validity B. Maximizing participant enjoyment C. Increasing the sample size **Answer:** Ruling out threats to research validity (A) Research validity is crucial, and experimental control focuses on ruling out potential threats to it. **2. What is an extraneous variable?** A. The same as the dependent variable B. A variable that is the main focus of the experiment C. A variable that can affect the results if not controlled **Answer:** A variable that can affect the results if not controlled (C) Extraneous variables are those that are not of primary interest but can confound results if not managed. **3. What is the primary benefit of Random Assignment to Groups in experimental research?** A\) It eliminates the need for a control group. B\) It ensures that groups are similar before the treatment. C\) It only allows for qualitative data collection. **Answer:** It ensures that groups are similar before the treatment (B) The primary benefit of random assignment to groups in experimental research is that it ensures groups are similar before the treatment. **4. Fill in the blank:** \_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_ is a control procedure in which the order of conditions is randomized, with each condition being presented once before any condition is repeated. What is this control procedure called? **Answer:** Block Randomization **5. What is one potential problem when using subjects as their own control?** A. Practice effects B. Enhanced control over extraneous variables C. Elimination of chance effects **Answer:** Practice effects (A) Practice effects can bias results when participants repeatedly experience conditions. **6. Fill in the Blank**: \_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_ is a technique used in experimental design to manage order effects in a repeated measures design by varying the sequence of conditions for participants. **Answer:** Counterbalancing **7. What is one suggestion for reducing experimenter bias in a study?** A. Increasing the number of participants B. Involving more than one experimenter C. Using longer instructions for participants **Answer:** Involving more than one experimenter (B) Involving more than one experimenter can help minimize individual biases**.** **8. Which of the following is an example of a systematic error in research?** A. Double-blind procedures B. Recording data incorrectly C. Participant misunderstanding instructions **Answer:** Recording data incorrectly (B) Recording data incorrectly represents a systematic error in data analysis. **9. What is a characteristic of double-blind procedures?** A. Neither the experimenter nor the participants know the expected outcomes B. Participants are informed about the expected outcomes C. Only the experimenter knows the hypothesis **Answer:** Neither the experimenter nor the participants know the expected outcomes (A) Double-blind procedures keep both parties unaware of the expected outcomes, reducing bias. **10. What is a moderator variable?** A. A control variable that remains constant throughout the experiment B. A variable that has no effect on the outcome of the study C. A variable that is manipulated to observe its effect **Answer:** A variable that influences the relationship between the independent and dependent variables (D) A moderator variable influences the relationship between the IV and DV. MGMT-6353 Research Design Midterm Questions Topic 4 ***Answers keyed in bold; includes page number for validation*** 1. Of the following, which is NOT one of the defining characteristics of experimental design? a. Random Assignment b. **Contiguity (Lecture Notes -- Page 20)** c. Manipulation 2. What type of analysis could be used if pretest or baseline results differ? d. **ANCOVA (Lecture Notes -- Page 22)** e. Check the equivalence of the groups f. Reset the baseline 3. In a within-subjects design participants g. Are randomly assigned to conditions h. Are selected to create equivalency i. **Experience every condition of the experiment (Lecture Notes -- Page 24)** 4. In a between-subject design participants j. Are randomly assigned to conditions k. Experience every condition of the experiment l. **Experience only one condition of the experiment (Lecture Notes -- Page 24)** 5. Which of the following is a disadvantage of a between-subjects design? m. Dependability of treatment effects n. **Equivalency is less assured (Lecture Notes -- Page 24)** o. Effects of repeated testing 6. In experimental design a control is defined as all except for which of the following? p. **Participants not exposed to the experimental manipulation (Lecture Notes -- Page 25)** q. Any means used to rule out possible threats r. Techniques used to eliminate or hold constant the effects of extraneous variables 7. Which of the following are the three conditions that must be met to infer cause? s. Contiguity, Constant Conjunction, and Validity t. Contiguity, Validity, and Temporal precedence u. **Contiguity, Temporal Precedence, and Constant Conjunction (Lecture Notes -- Page 26)** 8. An advantage of experimental design of other research designs is that v. Experimental design is usually more cost effective than other design methods w. **Experimental design permits the researcher to make causal inferences (Lecture Notes -- Page 26)** x. Experimental design permits the use of correlational methods to infer causal relationships 9. What is a major distinction between lab and field experiments? y. Lab settings are less controllable than field settings z. Field experiments allow for more precise measurement of behavior a. **Field experiments typically employ a real-life setting (Lecture Notes -- Page 32)** 10. Which of the following is an advantage of a lab experiment? b. **Measurement of behavior is very precise (Lecture Notes -- Page 32)** c. Results are highly generalizable d. Very realistic 11. What is a limitation associated with field experiments? e. Suggestions of causal inference are possible f. **Individuals or groups may decline to participate (Lecture Notes -- Page 33)** g. Some variables may have a weaker impact than in a lab experiment Cohort XIII Mid-term questions 10/05/2024 1. What type of Q-experimental Design has both experimental and control groups? a. Experimentally Controlled Design b. **Nonequivalent Control Group Design** c. Randomized Control Design 2. What is the most significant difference between true experimental design and quasi-experimental design? d. **How groups are assigned randomly** e. How variables are manipulated f. How large the control group 3. Which of the following is an example of a Delayed Control Group Design? g. Different groups are tested over the same time period. h. **Different groups are tested at different times.** i. Different groups are tested within-subject and between-subject. 4. Uncontrolled variables in Q-experimental designs can: j. **Reduce internal validity** k. Reduce external validity l. Reduce both internal and external validity 5. Non-equivalent control group designs present what main problem? m. **Results from non-equivalent groups are compared.** n. They produce non-equivalent statistical outcomes. o. Control groups have non-equivalent motivations. 6. Which of the following is not a characteristics of Mixed Factorial Designs? p. They incorporate within-subject and between-subject variables q. They use a combination of state and trait measures. r. **Interaction effects cannot be analyzed.** 7. Time-Series Experimental Designs typically: s. **Examine within-group trends before and after treatment.** t. Focus on a single trait characteristic across subjects. u. Compare measurements from multiple groups simultaneously. 8. Which of the following affects the interpretation of results from a Delayed Control Group Design? v. Merged groups complications. w. Simultaneous assessment of multiple variables. x. **Potential time-based biases.** 9. Repeated Treatment Designs [.] y. compare different groups between multiple treatments. z. **measure subjects' responses before and after repeated treatments.** a. repeatedly compare measurements of untreated subjects. 10. [ ] allows the same group to be compared over time. b. Mixed Factorial Designs c. Repeated Treatment Designs d. **Interrupted Time-Series Designs** 11. Mixed Factorial Designs [.] e. **have a within-subject variable and a between-subject variable.** f. allow within group comparisons over time. g. compare non-interrelated comparisons between groups. Dr. Edwards Mid Term Fall 2024 Topic 2: Research Methods: Variables and Measurement **ANSWER ARE IN BOLD** 1\. What does the Phi coefficient measure? 2\. Which option correctly describes a quantitative variable? 3\. What is essential for scientifically studying a construct? 4\. What is an example of a nominal scale? 5\. Which of the following scales includes a true zero point? 6\. Which measurement scale indicates how far apart objects are with respect to an attribute? 7\. Reliability in measurement refers to what aspect? 8\. What is a common example of an ordinal scale? 9\. For a measurement to be useful in science, it must have both: 10\. What defines an independent variable? 11\. Which statistical procedure is most appropriate for two continuous variables? 12\. What is a key characteristic of a dependent variable? 13\. Which of these describes a continuous variable? 14\. Which of the following methods is used to generate artificially discrete variables? 15\. What does convergent validity indicate? 16\. What is the main purpose of the multi-trait/multi-method matrix? 17\. Which type of validity assesses if different measures of different constructs are unrelated? 18\. What does face validity relate to? 19\. In the multi-trait/multi-method matrix, A and B represent which type of validity? 20\. What is the difference between criterion-related validities (concurrent, predictive, postdictive have to do with the \_\_\_\_\_\_\_\_\_\_\_\_\_\_\_ in the collection of criterion? A. Content **B. Time Frame** C. Measures