COMD 6230 Exam 1 Study Guide PDF

COMD 6230 - F23 Exam 1 Study Guide The closed-book test will be composed of multiple choice, matching, and short answer questions related to the questions in this guide. You do not need to memorize the contents of studies discussed in class. Look them up when answering the study guide, but for the e...

COMD 6230 - F23 Exam 1 Study Guide The closed-book test will be composed of multiple choice, matching, and short answer questions related to the questions in this guide. You do not need to memorize the contents of studies discussed in class. Look them up when answering the study guide, but for the exam, study information needed to answer questions will be provided at that time. Test is 100 minutes. [[https://quizlet.com/839541722/comd-6230-reserch-exam-1-flash-cards/?i=1ws4fy&x=1jqt]](https://quizlet.com/839541722/comd-6230-reserch-exam-1-flash-cards/?i=1ws4fy&x=1jqt) **[Scientific Method and Communication]** **1. What is the scientific method?** - Science comes from the Latin word meaning "to know" - Systematic testing of theory-based hypothesis about presumed relations among phenomena - Collection of measurable data through observation and experiment - Rational analysis of collected data to draw conclusions - Generate explanations and determine the probability of "truth" - Build knowledge: construct, replicate, extend, renovate - Systematic, controlled, empirical, rational, amoral, public, critical investigation - The scientific method initially involves recognizing that there are problems that can be evaluated objectively and then explained by collecting data through experiment or observation. Finally, conclusions are drawn by critically analyzing the data that was collected: Ask a question, Research, Form a hypothesis, Make a prediction, Conduct an experiment, Analyze the results, Draw conclusions **2. Why is the scientific method also called hypothetico-deductive?** - You're seeking knowledge through a process of testing general concepts through specific occurrences - 1\. Use your experience - 2\. Construct a general explanation or principle - 3\. Deduce a specific prediction or hypothesis - 4\. Test hypothesis to support or refute principle - Hypothetico: A hypothesis or theory is proposed for testing - Deductive: Test consequences are drawn from the hypotheses **3. What does it mean that the scientific method is probabilistic?** - It's different than proof - Scientific method and "research truth" is: - Probably true - Never know absolutely for sure - Highly confident likely to be true - Vague answers likelier truer than precise answers - A probabilistic method is a technique based on probability **4. Why does your instructor avoid the terms prove or proof in discussing research findings?** - Proofs are unquestionable - Proofs suggest predetermined answers - Science collects evidence for/against - Science should not be conducted in order to "confirm" an idea or opinion. When conducting research, refuting a claim is always a possibility. When research is done to "prove" something, the researchers may already be convinced of what the outcome will be. - We could say "strong evidence" or "probably true" **5. How does the attitude of research users affect how they make decisions based on the evidence?** - When research is done to "prove" something, the researchers may already be convinced of what the outcome will be. - "It is only through contradictory or disconfirming evidence that a scientific discipline is able to correct mistakes, misconceptions, or inaccuracies. This process, when combined with a receptive attitude to new ideas and a willingness to change, lies at the heart of a scientific approach to knowledge.... The goal of science is not to prove that something is correct but to determine what is true." **6. Your textbook says the method of intuition involves pure rationalism. What is the difference between intuition and rational thought?** - Rationalism states that knowledge is gained through deductive reasoning and logical thought. - Intuition relies on reason that is based on prior self-evident assumptions. Little or no consideration is given to the role of experience in the acquisition of knowledge. **7. Identify four features that signal a claim is based on junk/pseudo science.** 1. Little, if any, reliance on the existing scientific knowledge base 2. Vaguely defined, unsystematic, and irreproducible testing of ideas 3. Contradictory evidence is ignored, dismissed, or explained away 4. Over-reliance on personal experience, testimonials, and anecdotes 5. Acceptance of mysterious causes and beliefs, with little concern for identifying tangible underlying mechanisms 6. Lack of independent scientific review or the replication of results by others 7. No modification of claims or practices in the face of contradictory evidence 8. Support is based more on belief and faith than on objective evidence. 9. Use of your own jargon. 10. Extraordinary claims made. **8. Why should we be explicit and detailed in reporting our treatment and testing procedures?** - All potential conflicts of interest explicitly addressed - Legal reasons like participant consent - enable more robust peer-review; reviewers/editors can understand what authors did investigation - Need to be detailed so treatment can dependably be implemented and outcomes can be dependably measured. - So that others can replicate the study (helps with generalizability). **9. What are two reasons why citations are provided for claims in scientific writing?** - Provide a foundation on which the claim is based - In evaluating the rationale for study, reader determines whether premises are accurate/persuasive in support of researcher's propositions. - Makes research more credible - Tells where the information is from; give credit **10. What is a peer reviewed journal and how does peer review help assure quality of published studies?** - They publish good science - Written by experts and reviewed by several other experts in the field before the article is published in the journal in order to ensure the article's quality - More valid, reach reasonable conclusions - The text references The American Journal of Speech-Language Pathology. This editorial staff includes one editor-in-chief and 10 editors with expertise in certain subject areas. There are an additional 45 editors that have agreed to review 8-10 articles a year. The editor-in-chief establishes the editorial policy, which must adhere to ASHA guidelines. Once a manuscript is received, an editor is assigned to serve as a reviewer and oversee the reviewing process. The editor forwards the manuscript to two editorial board consultants, who recommend one of four things: accept for publication as is, accept contingent on the author agreeing to make certain revisions, defer a decision pending major revisions and an additional review by two different editorial consultants, or reject outright. The editor-in-chief makes the final decision to accept or reject the manuscript. If it is rejected, the author receives an explanation as to why the manuscript is not publishable. According to the text, "manuscripts are subjected to a "blind" review in which reviewers are ostensibly unaware of the identity of the author." - Author-submitted ms evaluated by expert peers and revised by author until satisfactory for publication - Helps maintain quality investigation and reporting - Guards against poor research, trivial problems, unclear writing. *Slide 12* Best Primary source is a top tier peer-reviewed published journal. **[Question and Control]** **1. What are theories and why are they helpful?** A theory is a formal representation of the data reduced to a minimal number of terms used to identify and outline cause and effect relationships. By coherently summarizing and organizing existing knowledge, theories establish a framework from which meaningful generalizations can be made. - Statements formulated to explain phenomena are called theories. A scientific theory is established through empirical and rational inquiry. - By coherently summarizing and organizing existing knowledge, theories establish a framework from which meaningful generalizations can be made. **2. What are three standards for judging the strength of a theory?** Five essential factors that can be used to judge the strength of a theory are (a)**accountability**- the ability of a theory to account for most of the existing data within its domain (b) **explanatory relevance**- explanation for a phenomenon provided by a theory must offer good grounds for believing that the phenomenon would occur under the specified conditions of the theory(c) **testability-** potential to be falsified (d) being **predictive**- new findings beyond that used to develop the theory (e) parsimonious -it should adopt the fewest and/or simplest set of assumptions in the interpretation of data (Occam\'s razor) **3. How do theories differ from hypotheses?** Theories attempt to explain problems. Testable version of a theory is a hypothesis, Hypotheses are proposed answers to testable research questions. **4. How does a study purpose differ from its research question?** Research purpose is a clear statement of what you are attempting to achieve through your research. The research question is a specific concern you will answer through your research. **5. Identify PICO elements of one of the research questions in Ukrainetz et al. (2009)** **Population-** At risk kindergarteners **Intervention-** concentrated schedule of treatment **Comparison-** dispersed schedule of treatment\`**Outcome-** show different immediate or final outcomes in phonemic awareness. **6. Rewrite a Ukrainetz et al. (2009) question to include all four PICO elements in a single question** **statement in three ways:** **a. What is the effect I O P C?** **b. What is the effect O P I C?** **c. What is the effect P O I C?** **7. What is a variable?** Variable is a feature of phenomenon that can vary or change - a variable is a measurable characteristic that can have more than one value **8. Why do we seek experimental control of variables?** - To have good internal validity. - Experimental control allows researchers to establish causal relationships between variables and reduce influence of confounding variables. When variables are controlled, experiments can be replicated by other researchers to verify the results. - Some of experimental controls: - Participant assignment - What tx the conditions received - What schedule the conditions received - Who taught the conditions - Tester preparation - Instructor preparation - Tx preparation - Performance checks **9. What are each of the following types of variables:** **a. Independent & dependent-** Independent variable is the presumed cause of the dependent variable, the presumed effect. Independent variables are conditions that cause changes in behavior. Dependent variables can be viewed as the behavior that is changed. **b. Extraneous-** intervening factors called a nuisance. Extraneous variables mediate the relationship between the independent and dependent variables.Extraneous variables may be responsible for the changes in the dependent variable, or may negate, moderate or even enhance the effect of the independent variable on the dependent variable. **c. Confounding & controlled-** When extraneous variables influence the relationship between independent and dependent variables, the experimental results become equivocal and are thus said to have confounded the outcomes.When extraneous factors are recognized and kept constant so as to minimize their effects on the outcome are called control variables **d. Predictor & predicted-** variables studied to determine relationship between them (intentional variation occurring, but which causes which is not clear, is it possible another variable causes joint variation). Examines how strongly and in what ways 2+ variables are related or associated. Example: Does age predict height? Does height predict age? **e. Active & attribute-** Whether can be manipulated in participant by investigators. Attribute is also called *organismic*. Attribute - can't change it (male vs female)(ELL vs ENGL). Active variables can be manipulated (intensity of treatment). **f. Categorical & continuous-** measured as present or absent; measured on a continuum of at least rank order. Example: Stuttering vs. not stuttering. **10. Identify one example of each type of variable from Ukrainetz et al. (2009).** Extraneous - the children learned how to learn because they are young. Independent- intensity Dependent- increase phonemic awareness **11. What is internal validity for an experimental versus a descriptive study?** Internal validity for an experimental study - sufficient control of extraneous variables and attributes changes in the dependent variable to independent variable. Internal validity for a descriptive study- sufficient control of variables and presents a convincing description of what occurred. **12. What does it mean to say a study design/method is \"tight\" versus \"full of holes\"?** Continuum of Internal validity- Is study well constructed enough to be an accurate test of research questions (a) procedures designed to test hypothesis(b) aim for well structured study with good internal validity(c)choose the right design for study question(d) plan procedures as well as possible given resources. Tight - well structured, carefully planned, executed with precision and control, implies methodology, robust with clear objections and hypotheses, likely to have minimized bias, confounding variables, and high internal validity Full of holes - plagued by significant methodological issues, various weaknesses, poorly defined research questions, various biases, low internal validity **13. What does it mean to say internal validity \"rules out competing explanations\"?** Critical aspect of research design, extent to which a study's design and execution support the conclusion that the observed effects of relationships are indeed due to the independent variables being manipulate or studied rather than being the result of other factors or explanations, study accurately measures what it's supposed to measure **14. Explain the ten potential threats to internal validity discussed in class (not \#10 subtypes).** **15. What are a priori vs. post hoc questions? Why are post hoc answers less certain?** **Priori questions** are answered by studies designed to answer them. Planned design, procedures, measures, participants for planned questions and hypotheses. Scientific method and hypothetico-deductive model. **Post hoc questions** are new questions on data after study planned and executed. They are less certain because the study was not controlled to answer the new questions. You need to be clear about post hoc questions and keep conclusions about findings tentative. **[Research Design]** **1. Explain these research design features:** **a. Single/group** - Single: - Performance of individual case examined - case= participant/subject or even institution/community - Average or pattern of performance across data points - Error controlled with many data points on individual performance - Group: - Performance of collected of cases examined - Average or pattern across group data points - Error controlled by having many participants **b. Between/within** - Between: - Comparing responses from different people or groups - Within: - Comparing multiple responses from one person or group **c. Retrospective/prospective** - Retrospective: - Looking back at preexisting data - Prospective: - Plan a study with experimental control to obtain data **d. Longitudinal/cross-sectional** - Longitudinal: - Measures on one group or person over long period of time - Benefit: can see how they've progressed over time - Cross sectional: - Measures on different groups at different ages or points in process **e. Relations/differences** - Relationship: - correlation/associations among two or more variables - Differences: - Dissimilarity among dependent variable results **f. Descriptive/experimental** - Descriptive research designs: - Describe sample to generalize to population - Systematic observation and data generation - Limited manipulation/control of environment - Size of sample versus level of detail - Labels: descriptive, observational, pre-post, non-experimental, pre-experimental - Example: - Topic: speech development - question: speech development of 2-5 year old children? - Representative sample of participants and speech sounds? - Focus on describing or comparing groups - Limited experimental control and manipulation - Types of descriptive designs: - 1\. Relationship descriptive designs - Direction, strength, nature of relation - 2\. Developmental - Attribute variables no manipulated, semi-longitudinal or cross-sectional, tracks change over age in dependent variable - 3\. pre/post design - Manipulation and systematic data collection, compare before/after for many participants, no comparison condition to control for time, attention or life variables - 4\. Survey research - Large number of participants remotely accessed, randomly sampled and stratified to match population, wording is important, analyzed through % responding this way or that - 5\. Epidemiological - Determining probability of events, extremely large samples of low incidence/prevalence, pre-existing groups, many variables must be controlled - 6\. Comparative descriptive designs - Compare two pre-exisiting groups on an attribute variable and judge reasons, not manipulating the attribute, try to make the groups as similar as possible - Experimental designs: - Manipulation of independent variables - Aim is cause-effect and predictions - Designs vary on degree of experimental control and internal validity - Critical feature: how participation in treatment and alternate tx or control conditions determined - Quasi experimental designs - 1\. Non-equivalent control groups - Non-randomized participant assignment procedures - Pre-existing groups and sequentially formed groups - Manipulation of independent variable and effort for strong experimental control - Descriptive-comparative designs with tx manipulation - 2\. Time series - Participants receive all conditions - Manipulation of independent variable and effort for strong experimental control - Single or few participants with many data points to track performances over baseline and tx time - True experimental designs - Randomized between-group participant assignment - Manipulation of independent variable - High experimental control and internal validity - Classic = randomized pretest-posttest control-group design - Variety of between- and within-group designs - Randomized clinical trials - Gold standard for tx research - Large scale study, very high experimental control and quality **2. Identify two research design features for Ukrainetz et al. (2009) and for the study examples discussed in class (See Study Examples in Canvas Files).** 1. Windsor (1994) - Longitudinal design 2. Ukrainetz et al. (2009) - Experimental design 3. Peterson et al. (2014) - Single subject multiple baseline design. **3. What kind of design is suggested in the following research questions:** **a. Does X cause Y?** - Experimental design **b. Is X positively associated with Y?** - Correlational design **c. How does X appear over time?** - Longitudinal design **4. What is an interaction effect between two independent variables?** An interaction effect happens when one independent variable has a different outcome depending on the values of another independent variable. It is the simultaneous effect of the independent variable and the parameter. **5. What are the features of the these main types of group designs** **a. Developmental**: attribute variables not manipulated, track change over age in dependent variable, relations or differences among variables, should be longitudinal but cross-sectional sometimes, semi-longitudinal. **b. Correlational**: how the relationships are described. It's a concept and a statistic. Bivariate scattergram graphs are helpful in visualizing relationships. **c. Pre/post:** manipulation and systematic data collection, compare before/after for many participants, no comparison condition to control for time attention or life variables, seems like treatment caused effect but insufficient control to be sure **d. Comparative**: comparing two pre-existing groups on an attribute variable and judge reasons, not manipulating the attribute, try to make the groups as similar as possible other than attribute and make cautious conclusions. Difficulty is that attributes are often complex and associated with many other differences. **e. Quasi-experimental:** - 1\. Non-equivalent control groups - Non-randomized participant assignment procedures - Pre-existing groups and sequentially formed groups - Manipulation of independent variable and effort for strong experimental control - Descriptive-comparative designs with tx manipulation - 2\. Time series - Participants receive all conditions - Manipulation of independent variable and effort for strong experimental control - Single or few participants with many data points to track performances over baseline and tx time **f. Experimental:** true experimental designs are randomized between group participant assignments. There's manipulation of an independent variable, high experimental control and internal validity. They use randomized clinical trials (which are the gold standard for treatment research, large-scale studies, high experimental control and quality). There\'s a variety of between and within group designs. **6. Does use of convenience sampling affect internal or external validity more?** External because it will be harder to generalize based on convenience sampling. **7. What study design uses randomized-stratified participant selection?** - Survey or epidemiological research study designs **8. What did Brandel and Loeb (2011) do to achieve a representative sample?** Coverage error occurs when not everyone in a population has an equal opportunity to participate in a survey. This was minimized by randomly selecting the participants from all ASHA-certified SLPs. Another potential concern is sampling error, which happens when a person surveys a portion of the desired population rather than everyone. According to the 2009 ASHA membership data. There were 59,980 SLPs who reported being employed in an educational school. Using these data and a formula for obtaining a representative sample the study had a 95% confidence level. **9. How does participant assignment occur in the following procedures** **a. Pre-existing: already exists in society before the experiment** **b. Sequential:** when a participant arrives is how they are put in the group. **c. Randomized: each participant equally likely to be placed in each condition.** **d. Balanced groups - matching on distribution of characteristics that are important.** **e. Matched pairs:** characteristics of participants are matched and put into different groups **10. Why can purely randomized assignment be problematic in small sample studies?** **11. How did Ukrainetz et al. (2009) combine randomized and balanced participant assignment?** They first created three groups from a coded list without the participant names. The groups were then balanced for age, English learned status and DIBELS intensive level. After creating these balanced group they were then randomly assigned each group to one of the three treatment conditions (CP=concentrated phonemic awareness, DP=dispersed phonemic awareness, CON= control group) **12. Can the effects of treatment on preschool stuttering recovery be determined in the Yairi and Ambrose (1999)? Why or why not?** No because it is a longitudinal study which only observes and not cause effect, no control, too many variables that you cannot decide if it actually worked. **13. What design is Le et al. (2014) and why does the Abstract report predictions rather than** **causes/effects?** The abstract reports predictions rather than cause/effects because the volumetric measures had ***only one significant correlation with the discourse measures***. Findings from regression analyses were analogous(comparable) but revealed several models that [approached significance]. ***Findings suggest that an overall measure of brain damage may be more predictive of general cognitive status than of narrative discourse ability.*** Atrophy measures in specific brain regions may be more informative **14. What is the difference between a "business-as-usual" and an active control condition and which is better for experimental control? Relate this to Elman and Bernstein-Ellis (1999).** In the article by Elman and Bernstein-Ellis (1999) employed an active control condition. First they randomly assigned participants to one of two conditions- Immediate treatment and Deferred treatment. Those who had been assigned to the deferred treatment did not carry on with 'business-as-usual'. In order to control for the effect of social contact and to ensure that none of them were isolated, the DT participants attended 3 or more hours weekly of social group activities of their choice. DT participants were reassessed on all measures following the socialization period just before their deferred treatment.By using an active control measure the researchers would be more likely able to attribute their group communication treatment as effective when able to rule out the known activities the participants had participated in. Having preselected activities would be more doable to control for than many business as usual activies. Active control = give them something (attention, placebo, other treatment, etc.) Business as usual = nothing is given. Business-as-usual has less experimental control. **15. What is a deferred treatment control condition and how does it address each of scientific control and ethics?** Ethics: The deferred treatment design avoids withholding treatment from the control group. **16. How does a comparison condition differ from a control condition, and which gives stronger internal validity?** Control conditions give stronger internal validity. Comparison condition uses pre-existing data as the control, and control condition has a more carefully chosen control group **17. How is a within-group repeated measures design used to compare two treatments? Why is this better than a between-group randomized assignment design?** **18. What major source of error can occur in a within-group repeated measures design and how can it be controlled?** Problem with within-group is potential for treatment order effects\... solve by randomizing which order you give the treatment in (wash out period= the time between necessary to make sure there isn\'t carryover) **19. Why are single-subject designs also called small N designs?** Single-subject designs may be applied to only one subject or to a small number of subjects who are evaluated as separate individuals rather than as members of a larger group to be averaged together. Therefore, it's important to keep in mind that studies that implement a single-subject or quantitative case-study design do not necessarily employ just one subject. As such, what are commonly referred to as single-subject designs are also known as small-N designs, where N represents the "number" of subjects. Of course, when only a single subject is studied, N = 1. **20. What study design is Windsor et al. (1994) and why can this design not answer whether treatment improved performance?** **21. How was Meadan et al. (2006) a within-subject comparative design study?** The two young boys with Autism were compared to one another, there was no control group. The data comparison occurs within the group study of participants and each participant serves as their own baseline. **22. What are two examples of experimental control exerted in Meadan et al. (2006)?** Experimental control was exerted as the researchers systematically considered the variables being examined and the effects of different combinations of variables such as percentage of repair across activities and the topography of repair within activities. To do this, scripted protocols were followed to create structured settings (left side p.61) **23. What does it mean that a participant is their own control in a single-subject experimental design?** The participant is compared to themselves **24. What is a time series experimental design?** quasi-experiment Rather than using a single pretest and a single posttest with a large number of subjects, time-series designs employ repeated measurements of the dependent variable over an extended period of time **25. How are time series phases the same as and different from conditions?** Time series phases and conditions serve to categorize different segments of a study, they have distinct conceptual foundations and are typically used in different types of research designs. Time series phases are used to study changes over time, whereas conditions are used to compare the effects of different levels of the independent variable. **26. What are the minimum number of data points in a baseline phase and why is more better?** 3+ data points per phase, 3+ for stagger, 2+participants/behaviors. Better if data taken until baseline is stable and if many data points within phases in order to provide sufficient data points to show a clear pattern across phases. More is better because it adds experimental control and thus greater internal validity. **27. Explain what happens in the following time single-subject experimental designs:** **28. In an ABAB design, what is the difference between the second A phase and a maintenance phase?** **Why does a researcher want to see performance drop in the second A phase?** **29. What are multiple baseline across behaviors and multiple baseline across subject designs?** Multiple baseline: implementing an intervention across different behaviors of the same individual, one behavior at a time, while keeping other behaviors as baselines. Multiple baseline across subjects: intervention is introduced to different individuals (subjects) one at a time, while keeping the others as baselines. **30. Using Petersen et al. (2014), explain how two features of the baseline design give confidence that (a) practice and (b) extraneous life events were not responsible for improved performance.** **31. In a multiple baseline across behavior design, what can you conclude about treatment efficacy if the control behavior starts improving at the same time as the treatment behavior?** There could be many other factors contributing to the change in behavior, other than treatment. Was the design implemented correctly? Has the treatment generalized across behaviors? Can external factors (ex: change in environment, routine, socialization) be influencing the behaviors? **32. Why do we do graphical analysis of the full pattern of the data in a single-subject study like Petersen et al. (2014), instead of just looking at the first baseline and last treatment data points?** So we can see growth or no growth over time. **33. Explain possibility versus probability in relation to single-subject and group designs.** [Single-subject design] Possibility: refers to the potential for an observed change in behavior to be attributed to the intervention. It acknowledges that the observed change may not be solely due to the intervention, as other factors could be influencing the behavior. "This treatment can work for someone like this person." Single-subject design = possibly; Group design = Probably Probability: refers to the likelihood that the observed change in behavior is a result of the intervention and not due to chance or other extraneous variables. [Group design] Possibility: refers to the potential for a relationship or difference observed in the group to exist in the larger population. Probability: refers to the likelihood that the observed relationship or difference in the sample accurately represents a real, systematic relationship or difference in the larger population. "This treatment will probably work for a lot of people." **[Methods ]** **1. What are the three principles of fair treatment for research participants?** a. Respect for persons (personhood): participants with diminished autonomy (prisoners, cognitive disability, children, pregnant women, etc.) are protected, participants are able to leave study at any time b. Beneficence: does study benefit whole population, benefit/risk to participants c. (Social) Justice: "the selection of individual populations or groups where you are looking at whether the benefits and burdens are spread across society to be fair and unbiased." --Ukrainetz **2. What is the difference between confidentiality and anonymity?** Confidentiality: who gets access to the files, privacy. Anonymity: researcher does not know identifying name of participants. **3. What are the two opposing forces that affect study participation explanation on a consent form?** Sufficiently detailed and precise to cover legal requirements vs. should be written in language appropriate to attention and comprehension level **4. How can a high attrition rate damage a study\'s internal and external validity?** Attrition = loss of participants from study. Internal validity: Attrition can introduce *selection bias*, where the characteristics of participants who drop out differ systematically from those who remain in the study. Also, the remaining sample may no longer be representative of the initial sample, potentially leading to skewed results. External validity: *Limited Generalizability*, where if a large portion of participants drop out, the remaining sample may not accurately represent the broader population from which it was drawn. This limits the ability to generalize findings to a larger population. **5. Did Ukrainetz et al. (2009) meet the rule-of-thumb minimum sample sizes for overall study, each** **condition, and each cell for the questions asked by the investigators?** No, she did not meet each cell criteria. - Factors that would require more participants than the rule of thumb: variability in what the participants can do. You will need a bigger sample to get past the variation. - Physiological studies: don't worry about having large samples **6. What are three factors that affect decisions about sample size in a group design.** **7. Why are homogeneous participant samples better than heterogeneous samples for internal validity?** Homogenous samples are better for finding and explaining differences, reduces variation within the group so mean performance is more representative, different levels of independent variable more likely to have predictable dependent variable outcomes, reduces competing explanations. Heterogeneous samples are more applicable to more people but also have less experimental control **8. What are two ways to investigate heterogeneous populations and still have good experimental** **Control?** 1. Systematic replication: do it again on a different sample (different age, different ethnicity, etc) 2. Enlarge the sample size and use subgroups (costs more money) **9. What type of design uses extremely large sample sizes and why?** 1. Survey research: composed to be representative of population, stratified and randomized 2. Epidemiological Research: when base rate is extremely low and many variables, may be examined over years, 10s of thousands of participants **10. Why are many data points collected for each participant in a single-subject design?** It allows you to see the trend in the data, reduces the impact of outliers **11. What is external validity?** Degree to which results can be generalized from the sample to population **12. What is the trade-off often faced between internal and external validity?** Better internal validity comes at the expense of external validity. Example: a controlled setting has better internal validity because external influences can be minimized. However, the external validity diminishes because a controlled environment is different from the outside world that does have external influencing factors. A solution is to conduct the research first in a controlled environment to establish the causal relationship, followed by a field experiment to analyze if the results hold in the real world. **13. Explain and give examples of ratio, ordinal, interval, and categorical data.** Categorical/Nominal: Mutually exclusive categories; named groupings. qualitative data that can be grouped into categories instead of being measured numerically. Example: area code, hometown, age, race, educational experience, type of disfluency, sex. Ratio: Equal intervals; true meaningful zero; plus properties of all other scales. Example: Age, weight, vowel, duration, number correct Ordinal: Ranked data; plus properties of Nominal. Example: sweetest-sourest, tallest-shortest, severity. Interval: Equal intervals; no meaningful zero; plus properties of Ordinal and Nominal. Example: temperature, acidity, IQ and other standard scores. **14. What is the difference between objective and subjective data? Which is more valid?** Objective: totally observable and measurable with no difference from inner or unmeasurable aspects of it. No disagreement on how we measure it, but doesn't mean it's true. Subjective: someone's opinion/ how they perceive something/ not fully measurable, up to interpretation. An opinion may not be measurable, but still may be true. Validity depends on the context **15. What did Brandel and Loeb (2011) do to check they had good quality survey questions and answers?** Initial versions of the questionnaire were modified as a result of input from a focus group of four ASHA-certified SLPs who were currently employed in schools. Following changes made based on the suggestions of the focus group, a pilot study with 77 SLPs (response rate of 42%) in northeast Kansas was conducted to evaluate the ability of the tool to answer the proposed research questions. **16. How did Yairi and Ambrose (1999) give confidence that their judgment of presence of stuttering for participant entry into the study was valid and reliable?** They included an inter-reliability measure that had two researchers determining the level of a participant's stuttering and did a point to point analysis to ensure reliability. **17. What does it mean to calibrate an instrument? What is observer calibration?** Instrument calibration: making sure the instruments you're using in an experiment/what not are "calibrated" and accurate. Calibrating an instrument means checking and adjusting its response so the output accurately corresponds to its input. This process ensures that the instrument is measuring within specified accuracy limits. Observer calibration: how accurate and how reliable are you as an observer who is measuring data. **18. What are two ways that Ukrainetz et al. (2009) determined the testing was reliable?** 1\. All testers were blind to conditions at pretesting. 2\. Those who served as instructors did not carry out mid- and posttesting. **19. For independent and consensus scoring, which is a measure of reliability and which is more about increasing accuracy?** Independent scoring is primarily a measure of reliability. The purpose of independent scoring is to assess the degree of agreement or consistency among raters. Consensus scoring is more about increasing accuracy rather than being a direct measure of reliability. Consensus scoring involves having multiple raters or observers discuss and collaborate to reach a collective agreement or consensus on the ratings **20. For reliability scoring, what is point-by-point agreement and why is that better than total score** **agreement?** Point-by-point agreement: comparing how each individual section/question/point relates to each other--do this to gather the 80% agreement score Total score agreement: if the total score of the two participants is agreeable Point-by-point is preferred because it helps identify specific areas of disagreement. **21. Calculate point-to-point agreement on a dataset like this (Yes = Scored as correct, No = Scored as incorrect):** 17 agreed/20 items = 85% -------------- ------- ------- ------- ------- ------- ------- ------- ------- ------- -------- -------- -------- -------- -------- -------- -------- -------- -------- -------- -------- **Item \#1** **1** **2** **3** **4** **5** **6** **7** **8** **9** **10** **11** **12** **13** **14** **15** **16** **17** **18** **19** **20** **Scorer 1** **Y** **Y** **Y** **Y** **Y** **Y** **Y** **N** **Y** **Y** **N** **Y** **Y** **Y** **N** **N** **N** **Y** **N** **Y** **Scorer 2** **Y** **Y** **Y** **Y** **N** **Y** **Y** **Y** **Y** **Y** **N** **Y** **Y** **Y** **Y** **N** **N** **Y** **N** **Y** -------------- ------- ------- ------- ------- ------- ------- ------- ------- ------- -------- -------- -------- -------- -------- -------- -------- -------- -------- -------- -------- **22. What is a rule-of-thumb minimum level of inter-rater agreement?** Minimum level of inter-rater agreement = greater than 80% **23. If scoring reliability is low, what can the researchers do to improve it?** If lower, retrain and simplify scoring system or explain how that's ok. **24. What is treatment fidelity and why is it important to report?** How closely a treatment is delivered or implemented as intended. It is important because it makes it more reliable for scoring, gathering and assessing data, and for replication. **25. What two ways did Ukrainetz et al. (2009) check the fidelity of their phonemic awareness treatment delivery?** Treatment fidelity was judged for content and schedule. Content: "observations showed that both instructors provided consistent treatment with explicit skill foci, repeated opportunities, and systematic scaffolding on the specified learning objections." Schedule: "number of treatment sessions and the ratio were lower than planned. However, sessions were equal in mean and range among three conditions."