Research Methods Final Notes PDF

Define experiment: A type of study designed to specifically answer the question of whether there is a casual relationship between two variables Changes in one variable (the independent variable) causes a change in another variable (referred to as a dependent variable); Define independent variable...

Define experiment: A type of study designed to specifically answer the question of whether there is a casual relationship between two variables Changes in one variable (the independent variable) causes a change in another variable (referred to as a dependent variable); Define independent variable: Variable that is being manipulated; Define dependent variable: Variable that is being measured; Define conditions: The different levels of the independent variable; Define extraneous variables Variables other than the independent and dependent variables Anything that varies in the context of a study other than the independent and dependent variables; Define control: Minimization of extraneous variables; Explain the manipulation of the independent variable: Manipulation refers to a change that creates a difference in exposure level Manipulation must involve an active intervention Sometimes it is not possible to manipulate a variable; Define and describe the two types of manipulation: Single factor two level design, a single independent variable with two conditions Single factor multi level design, one independent variable that is manipulated to produce more than two conditions; How are extraneous variables controlled: Extraneous variables are likely to have some effect on the dependent variables Influencing factor can make it difficult to separate the effect the independent variable has from the effect of the extraneous variables Important to control extraneous variables by holding them constant; Extraneous variables as “noise” and how to reduce the noise: The addition of random noise makes detecting the effect of the independent variable difficult Extraneous variables can be held constant to reduce this noise, this is done through conducting the investigation in the same location, with identical instructions, treating each one the same way, and limiting participants to one specific category; How are extraneous variables as confounding variables: Confounding variable is an extraneous variable that differs on average across levels of the independent variable; Define and describe control conditions: No-treatment control conditions, where participants are not given any treatment Placebo control condition, where a stimualted treatment that lacks any active ingredient or element that make it effective. A placebo effect describes receiving or perceiving an effect like the treatment; Describe between-subject experiments: Each participant is tested in only one condition Essential that participants are randomly assigned so groups are as similar as possible Advantage is that it is conceptually simpler and require less testing time per participant and avoids carryover effects without the need for counterbalancing; Define random assignment, the two criteria that must be met and block randomization: Using a random process to decide which participants are tested in which conditions Two criteria should be met, each participant has equal change of being assigned to each condition. Each participant is assigned to a condition independently of other participants. Block randomization describes that all the conditions occur once in the sequence before any of them is repeated. Within each of these blocks, the conditions occur in a random order. The sequence of conditions is usually generated before any participants are tested, and each new participant is assigned to the next condition in the sequence. This prevents unequal sample sizes; Describe matched groups: Participants in the various conditions are matched on the dependent variable or on some extraneous variable(s) prior to the manipulation of the independent variable Guarantees that the variables will not be confounded across the experimental conditions Helps control for extraneous variables and remove doubt regarding causation; Describe within-subject experiments: Each participant is tested under all conditions Provides maximum control of extraneous participant variables Makes it possible to use statistical procedures that remove the effect of these extraneous participant variables on the dependent variable and therefore, make data less “noisy” and the effect of the independent variable easier to detect Advantage of controlling extraneous participant variables which generally reduces noise in the data and makes it easier to detect any effect of the independent variable upon the dependent variable. Require fewer participants than between-subjects experiments to detect an effect of the same size; Define order effect: Participants’ responses in the various conditions that are affected by the order of conditions to which they were exposed; Carryover effect: An effect of being tested in one condition on participants’ behaviour in later conditions; Practice effect: Participants perform a task better in later conditions because they have had a chance to practice it; Fatigue effect: Participants perform a task worse in later conditions because they become tried or bored; Context effect: Being tested in one condition can also change how participants perceive stimuli or interpret their task in later conditions; Counterbalancing: Testing different participants in different orders; Complete counterbalancing: An equal number of participants complete each possible order of conditions; Random counterbalancing: Order of the conditions is randomly determined for each participant; Describe simultaneous within-subject designs: Used when participants make multiple responses in each condition Instead of testing each response in order, the entire set of questions can be presented in a sequence that mixes the types; Types of validities: Answers “is the study accurate Internal validity, external validity, construct validity, and statistical validity; Define internal validity: The extent to which a piece of evidence supports a claim about cause and effect, within the context of a particular study. Experiments are high in internal validity because the way they are conducted; Define external validity: The extent to which the results and conclusions of a study can be applied outside the context of the study and generalized across other situations, people, stimuli and times; Define construct validity: The degree to which a test measures what it claims, or reports, to be measuring; Define statistical validity: The extent to which the conclusions drawn from a statistical test are accurate and reliable; Describe participant recruitment: Subject pool, where an established group of people who have agreed to be contacted about participating in research studies Participants who are not in subject pools can be recruited by posting, publishing advertisements, or making personal appeals to groups; Side effects of not standardizing the procedure: If not standardized, it is easy to introduce extraneous variables during the procedure When there are multiple experimenters, the possibility of introducing extraneous variables is even greater Experimenter expectancy effect is about how participants “should” behave in the experiment. May knowingly or unknowingly bias the treatment group, double blinded studies help control for this by not revealing which group the participant is in to the experimenter; Explain record keeping: Experimental logs is a place for experimenter to write down comments about unusual occurrences or questions that come up Participant's identities should be kept confidential as possible Masters list should be kept separate in a secured/encrypted format; Explain manipulation check: It is a separate measure of the construct the researcher is trying to manipulate. The purpose is the confirm that the independent variable was successfully manipulated. Manipulation checks are often done at the end of the procedure to ensure that the effect of the manipulation lasted throughout the entire procedure and to avoid calling unnecessary attention to the manipulation Manipulation checks are particularly important when the results of the experiment turn out null. It can determine whether the null result is due to a real absence of an effect of the independent variable on the dependent variable, or if it is due to a problem with the manipulation of the independent variable; Explain pilot testing: Is a small scale study conducted to make sure that new procedure works as planned; List and explain the different types of research: Retrospective research, data were collected before the research questions were developed or the data is collecte and the research question is based on the avaliable data. Retrospective data includes case reports, case studies, case control and cohort studies Prospective research is when research questions and variables are determined before collecting data. Prospective research includes cohort studies and randomized controlled trials. Compilation research is a collection of data from a variety of studies and combines the large pool of data to answer the clinical question. Compilation research includes systematic reviews and metanalyses; Describe case report/study: A detailed description of a single case/patient Allows clinician to report a rare/unique clinical effect Provides no statistical comparison Not possible to make inferences about cause and effect Not possible to generalize the outcome of the cases with confidence; Describe case series: Collection of info gathered on a topic Patients with a known condition are given a similar treatment Observational Provides documentation on the situations when a new/complex intervention is used or when a new/complex condition is encountered; Describe case control: Observing a patient with a particular condition while observing like individual who do not have the condition Allow comparison of 2 like groups that differ Difference is the target treatment/intervention Comparisons based on an outcome (condition), not an intervention; Describe prospective cohort study: A group of individuals that share a common experience over a period of time All are free of condition in the beginning Separated into groups depending on exposure to risk factors associated with the condition; Describe retrospective cohort study: Identify groups based on exposure or intervention at some point in the past, and then follow the groups forward; Describe cohort studies: Take baseline measurements on individuals and track them over time Subjects are separated into two groups, those who experienced the particular injury and those who did not experience the particular injury. Tracks group of individuals over time to determine if an injury occurs; Describe randomized controlled trials (RCT): The gold standard of experimental research At least two groups with the same condition (experimental vs control) Randomly assigned to a group Experimental group receives an intervention Control group receives an alternate treatment/placebo/no treatment Both groups evaluated to determine any differences; Describe with patient randomized treatment order trials: Can provide the strongest evidence Patients receive all interventions under consideration in random order Patients serve as their own control Minimizes the influence of responsive differences between participants Minimized the number of needed (no comparison group); Define systematic review: Collects evidence from multiple research studies Gold standard for making clinical decisions; Define meta-analysis: Combines statistical information Similar to systematic reviews but focuses on a statistical process to combine results; Describe the hierarchy of evidence: There are levels of evidence which compares the article’s strength compared to others Doesn’t necessarily take clinical applicability into consideration, it considers validity, cost, easy of implementation, and reproducibility. Numeric rating system: 1-5 depicts validity of study CEBM and SORT levels of evidence and grades of recommendation; Describe CEBM grades of recommendation: Grades and recommends the level of confidence for the evidence to be incorporated into clinical practice A, B, C, D, or I Suggests how much the info should influence clinical practice; Describe SORT grades of recommendation: SORT is strength of recommendation taxonomy Two rating systems, levels to grade individual studies and grades to make clinical recommendations concerning bodies of evidence; Define non-experimental research: Research that lacks manipulation of an independent variable (not manipulating the manipulating variable) They just measure variables as they naturally occur They cannot prove that changes in independent variable causes differences in dependent variable; When to use non-experimental research: When the researcher has a specific research question or hypothesis about a casual relationship between two variables that that it is possible, feasible and ethical to manipulate the independent variable. When the research question or variable focuses on one single variable rather than the relationship between two variables. When the research question is pertains to a non-casual statistical relationship between variables. When the research question is about a casual relationship, but the independent variable cannot be manipulated or participants cannot be randomly assigned to conditions or orders of conditions for practical or ethical reasons. When the research question is broad or exploratory; Correlational research of non-experimental research: Researcher measures two variables with little or no attempt to control extraneous variables (a factor that can affect the outcomes) and then assess the relationship between them. It is non-experimental because it focuses on the statistical relationship between two variables but does not include the manipulation of the independent variable; Observational research of non-experimental research: Non-experimental because it focuses on making observations of behaviour in a natural or laboratory setting without manipulating anything; Internal validity: Internal validity is the extent to which the design of a study supports the conclusions that changes in the independent variable caused any observed differences in the dependent variable Experimental research tends to be the highest in internal validity because of manipulation and control Non-experimental research is lowest in internal validity because these designs fail to use manipulation or control Quasi-experimental research falls in the middle bc it contains some, but not all, of the features of a true experiment; What is correlational research: A type of non-experimental research where the researcher measures two variables (binary or continuous) and asses the correlation between them with little or no effort to control extraneous variables; Why correlational research: Researchers do not believe the statistical relationship is a casual one or are not interested in casual relationships The statistical relationship is casual, but researcher cannot manipulate the independent variable because it is impossible, impractical or unethical To establish the reliability and validity of measurements Often higher in external validity than experimental research Help to provide converging evidence for a theory; Data collection in correlation research: No manipulation of variables can occur Does not matter how the variables are measured Measurement can be quantitative or qualitative; Correlations between quantitative variables, scatter plots and Pearson’s correlation coefficient: Correlation can be graphed visually or calculated using statistics Scatter plots plot the relationship between two variables. When it is positive, higher scores on one variable tend to be associated with higher scores on the other. When it is negative, higher scores on one variable tend to be associated with lower scores on the other. Pearson’s correlation coefficient. Ranges from –1 to +1. Value of 0 means no correlation relationship between the two variables. Good measure for linear relationships, in which the points are best approximated with a straight line; Reasons for correlation does not imply causation: Directionality problem, two variables, X and Y, can be statistically related because X causes Y, or Y causes X (goes both ways) Third variable problem. Two variables X and Y can be statistically related not because X causes Y or because Y causes X, but because some third variable Z, causes both X and Y; Describe complex correlation, assessing relationships among multiple variables: Involves measuring several variables and then assessing the statistical relationships among them A correlation matrix can be used to show the correlation between every possible pair of variables; Factor analysis: Factor analysis can be used to study relationships among a large number of conceptually similar variables Factor analysis organizes the variables into smaller number of clusters, such that they are strongly correlated within each cluster but weakly correlated between clusters Underlying constructs are also known as factors Each cluster is then interpreted as multiple measures of the same underlying construct Factors are constructs that operate independently of each other; Explain regression: Regression can be used to make predictions about the value of one variable given the value of another variable Can be used to describe more complex relationships between more than two variables The variable that is used to make the prediction is referred to as the predictor variable and the variable that is being predicted is called the outcome variable or criterion variable. Regression equation, Y=b1X1. Y represents the person’s predicted score on the outcome variable, b1 represents the slop of the line depicting the relationship between the two variables and X1 represents the person’s score on the predictor variable. Simple regression describes when one variable predicts another Multiple regression describes when several variables predict one outcome variable. When a predictor variable contributes to the outcome over and above the contribution of another variable; Qualitative research: In depth information about relatively few people Conclusions are based on interpretations drawn by the investigator Global and exploratory; Quantitative research; Less depth information with larger samples Conclusions are based on statistical analysis Specific and focused; Data collection, interviews: Most common approach Can be unstructured or structured. In between are called semi-structured; Focus groups, interviews: Small groups of people who participate together in interviews focused on a particular topic or issue. Can sometimes bring out more info than one-on-one interviews; Data analysis approaches, grounded theory and theoretical narrative: Data analysis is the most distinguishing factor that separates qualitative and quantitative research Grounded theory, researchers start with the data and develop a theory or an interpretation that is “grounded in” those data Theoretical narrative focuses on the subjective experience of the participants and is usually supported by any direct quotations from the participants themselves; Describe observational research: Refers to several different types of non-experimental studies where behaviour is systematically observed and recorded Goal is to describe a variable or set of variables Goal is to obtain a snapshot of specific characteristics of an individual, group, or setting The data collected are often qualitative in nature but they can also be quantitative or mixed methods; Describe naturalistic observation: Researchers usually make their observations as unobtrusively as possible so that participants are not aware that they are being studied. Undisguised naturalistic observation is when the participants are made aware of the researcher’s presence and monitoring their behaviour Reactivity is when a measure changes the participant’s behaviour. A concern with this is that when people know they are being observed and studied, they may act differently than they normally would. Hawthrone effect is a type of reactivity; Describe participant observation: When researchers become active participants in the group or situation they are studying It is similar to naturalistic observation in that it involves oberving people’s behaviour in the environment in which it typically occurs Disguised participant observation is when the researcher pretends to be members of the group they are observing and conceal their identity as researchers Undisguised participant observation is when the researcher becomes a part of the group they are observing and disclose their identity; Describe structured observation: Investigator makes careful observations of one or more specific behaviours in a particular setting that is more structured than the settings used in naturalistic or participant observation The settings in which the observations are made may not be the natural setting (ie, the lab) Coding requires clearly defining a set of target behaviours. The observers categorize participants individually in terms of which behaviour they have engaged in, and the number of times engaged. Coding has interrater reliability issues and researchers are expected to demonstrate the interrater reliability of their coding procedure by having multiple rater code and the same behaviours independently and then showing that the different observers are in close agreement; Describe case studies: An in-depth examination of an individual Sometimes case studies are also completed on social units and events Case studies are useful because they provide detailed analysis and greater insights; Describe archival research: Involves analyzing archival data that have already been collected for some other purpose Content analysis is a research tool used to determine the presence of certain words, themes, or concepts within some given qualitative data What are the four elements to Guba’s model of trustworthiness: Truth value, applicability, consistency, neutrality; Describe truth value of Guba’s model of trustworthiness: Whether the researcher has established confidence in the truth of the findings for the subjects or informants and the context in which the study was undertaken Researchers need to focus on testing their findings against various groups from which the data were drawn, or persons who are familiar with the phenomenon being studied; Describe applicability of Guba’s model of trustworthiness: Degree to which the findings can be applied to other contexts and settings, or with other groups. The ability to generalize from the findings to larger populations. Two perspectives, generalizability not relevant in many studies, as they are naturalistic with few controlled variables. Transferability may be important, the extent to which findings fit into contexts outside the research context; Describe consistency of Guba’s model of trustworthiness: Whether the findings would be consistent if the inquiry were replicated with the same subjects, or in a similar context Since variability is expected in a perspective valuing individual differences, consistency is defined in terms of dependability; Describe neutrality of Guba’s model of trustworthiness: The freedom from bias in the research procedures and results The degree to which the findings are a function of solely of the informants and conditions of the research and not of other biases, motivations and perspectives Confirmability (reproducibility) may be a better indicator of neutrality; What are techniques for demonstrating credibility: Time sampling by interviewing participants at different time Triangulation by checking your results with multiple raters, or multiple measures Member checking by returning to your test group with your results, and asking if they seem applicable/correct Peer examination; What are techniques for demonstrating transferability: Comparison with demographic data to demonstrate that your sample conforms to population parameters for variables of interest Nominated sample by having an expert choose a sample of individuals that he/she considers to be representative; What are techniques for demonstrating dependability: Dense description that includes as much information from the focus groups as possible Stepwise replication, have a colleague work through your raw data, and see if he/she comes to the same conclusion Triangulation by checking your results with multiple raters, or multiple measures Peer examination Code-recode procedure, similar to stepwise replication, except you leave it alone for a few months, and then re-analyze it; What are the techniques for demonstrating confirmability: Triangulation by checking your results with multiple raters, or multiple measures Reflexivity by creating a journal to record thoughts, biases and impressions throughout the study. Guards against unintentional experimenter bias by being more aware of it; What are the strengths of qualitative research: Naturalistic Open and relatively unstructured Researcher as key data collection instrument Designed to collect descriptive data Orients to a more focused description Focuses on processes rather than product Focuses on participant perspectives to achieve deeper understanding of the data; What are the weaknesses of qualitative research: It is labour intensive Involves experience-based learning Uses a different set of methodological assumptions, making it more difficult to establish credibility of findings and different ways of determining clinical utility May be open to abuse due to the lack of formal rigor in data collection and data analysis; Critical evaluation of qualitative research: Expertise, is the researcher an expert in the research technique Research question, is the question answerable with the design proposed, are there other methods that might have been better Literature review, have other researchers answered questions like this one Context, is this technique appropriate for the research setting, is authentic assessment possible in this setting Sample, is there reason to expect that the participant group is representative Data collection and analysis, is there evidence of integrity in data gathering and analysis; Describe surveys: Surveys are correlational research, casualty may not be inferred Some survey research makes predictions through predictor variables and criterion variables Survey types include questionnaires, interviews and self reported diaries; Define constructs: This is the most important stage in survey design Used to understand differences between individuals and groups; Types of questions: Demographic questions Open-ended items, which have no specific set of alternative answers, critical incident technique and analyze with content analysis. Most often used in interviews. Tends to be subjective. Close-ended (restricted) items are a specific set of alternative answers, particularly open- ended items (“other” category), forced choice items and rating scales Should be collecting interval data, and only resort to ordinal data for questions about which a participant may be unsure about specifics; Define scales and the types of scales: Categorical scales (nominal), participant assigns self to a category and the categories must be mutually exclusive. Continuous scales (interval/ratio) Ranked scales (ordinal); Describe summative and cumulative scales: Summative is where each item contributes equally to the total score Cumulative is when each item represents an increasing amount of the attribute being measured; Describe the Likert scale: Rating scale used to measure people’s attitudes toward something by assessing their level of agreement with several statements about it Involves presenting people with both favourable and unfavourable statements Summative scaling method involving ranked values There is no clear consensus on the number of points on the scale The anchors, need to be clear, and ensure wording does not include bias The odd number of categories allow for the choice of neutrality; Describe semantic differential: Another summative scaling method Measures individual’s feelings about a particular construct by scaling between two extremes. Only the extreme anchors are labeled; Describe visual analog scale: Summative scaling method similar to semantic differential where only the extreme anchord are labeled Line of fixed length used for scale Not usually limited to feelings, the most frequent application is the description of pain; Explain the Guttmann Scale: Cumulative scaling method that is often employed to describe the functional limitations of patients Items in the scale are cumulative or hierarchical, once you disagree with one items, you must not agree on any further items; List the types of questions you should avoid asking in a survey: Jargon and slang Assumptions Absolutes Leading questions Low inference statements Double negatives Barnum statements, statements that are so general that mot people endorse them in the same direction Bias wording, when the question suggests the “answer” Double-barreled questions, asking two questions at once Overlapping categories; Ways to reduce social desirability bias; Catch trials, items that have no societal bias, but are infrequently endorsed by honest/attentive participants Include equal numbers of “positively” and “negatively” keyed terms Repeated items, to check for consistency; Describe Delphi survey: A survey in which participants are health-care practitioners, or experts in the field, this is to develop a consensus around a specific issue. Useful for establishing norms in clinical practice; Explain the cognitive model: Outlines the psychology process respondents must go through to interpret the question, retrieve relevant information from memory, form a tentative judgement, convert the tentative judgement into one of the response questions provided; Name elements that effect survey responses: Context effects, influencers are not related to the content of the item but to the context in which the item appears Item-order effect, the order in which the items are presented affects people’s responses Response-option effect, the response options themselves can influence responses, and the response options may amplify or reduce the interpretation of the response categories; Describe open-ended items: As a question and allow participants to answer in whatever way they choose Useful when the researchers do not know how participants might respond or when they want to avoid influencing their responses More qualitative in nature Easier for the researcher to write but take more time and effort for participant More difficult for the researcher to analyze Respondents are more likely to skip open-ended items because they take longer to answer. Best to use when the answer is unsure or the quantities which can easily be converted to categories later in the analysis; Describe closed-ended items: Ask a question and provide a set of response options for participants to choose from. Used when researchers have a good idea of the different responses that participants might make. More quantitative, used when researchers are interested in a well-defined variable or construct More difficult to write because they must include an appropriate set of response options. Quick and easy for the participants to complete Easy for researchers to analyze because the responses can be easily converted to numbers and entered into a spreadsheet; Describe rating scale: An ordered number of responses that participants must choose from Number of response options on a typical rating scale ranges from 3-11, 5-7 are most common. Five point scales are best for unipolar scales were only one construct is tested, such as frequency Seven point scales are best for bipolar scales where only one construct is tested, such as frequency. It is useful to offer an earlier dichotomous (Yes/No) question for bipolar questions that branches into an area of the scale (branching improves reliability and validity Scales often use numerical labels, it is best to only present verbal labels to the respondents but convert them to numerical values in the analyses; Explain the BRUSO model: Brief and to the point Relevant, avoid long, overly technical or unnecessary words Unambiguous, can be interpreted in only one way Specific, clear to respondents what their response should be about and clear to researchers what it is about Objective, do not reveal the researcher’s own opinions or lead participants to answer in a particular way; How to best format the survey: Every survey should have a written or spoken introduction that serves two basic functions, to encourage respondents to participate in the survey, and to establish informed consent Introduction should be followed by the questionnaire items Important to present clear instructions for completing the questionnaire The introduction is the point at which the respondents are usually most interested and least fatigued, so it is best to put the most important items first. Items should be grouped by topic or type Demographic items are presented last because they are least interesting to participants but also easy to answer. Survey should end with an expression of appreciation to the respondent; Describe one-group posttest only design: A treatment is implemented (or an independent variable is manipulated) and then a dependent variable is measured once after the treatment is implemented The weakest type of quasi-experimental design due to the lack of control or comparison group Results are frequently reported in the media and are often misinterpreted by the general population; Describe one group pretest-posttest design: Dependent variable is measured once before the treatment and once after implemented Much like a within-subject design, where participant is tested first under control condition then under treatment condition Also unlike within-subject experiment as the order of conditions is not counterbalanced because it typically is not possible for a participant to be tested in the treatment condition first and then in an “untreated” control condition If the average posttest score is better than the average pretest score, then it makes sense to conclude that the treatment might be responsible for the improvement, however, not with a high degree of certainty because there may be other explanations for why the posttest scores may have changed, such as history, maturation, testing, instrumentation, regression to the mean, spontaneous remission, etc; Explain interrupted time series design: A time series is a set of measurements taken at intervals over a period of time. It is like a pretest-posttest design in that it includes measurements of the dependent variable both before and after the treatment It is unlike the pretest-posttest design inn that it includes multiple pretest and posttest measurements; Explain posttest only nonequivalent groups design: Participants in one group are exposed to a treatment, a nonequivalent group is not exposed to the treatment, then the two groups are compared. Disadvantage is that without random assignment, there is still the possibility of other confounding variables that the researchers cannot control; Explain pretest-posttest nonequivalent groups design: There is a treatment group that is given in a pretest, recieves a treatment, then the posttest At the same time, there is no nonequivalent control group that is given a pretest, does not receive the treatment, and then is given a posttest The participants who recieved the treatment are then compared to the participants who did not If participants in this kind of design are randomly assigned to conditons, it becomes a true between groups experiment rather than a quasi experiment; Explain interrupted time-series design with nonequivalent groups: Involves taking a set of measurements at intervals over a period of time both before and after an intervention of interest in two or more nonequivalent groups; Pretest-posttest design with switching replication: Nonequivalent groups are administered a pretest of the dependent variable, then one group receives a treatment while nonequivalent control group does not. The dependent variable is assessed again, and then the treatment is added to the control group, and finally the dependent variable is assessed one last time Strengths is that it includes built-in replication, and provides more control over history effects as it becomes rather unlikely that some outside event would perfectly coincide with the introduction of the treatment in the first group and nonequivalent with delayed introduction of the treatment in second group; Switching replication with treatment removal design: The treatment is removed from the first group when it is added to the second group Demonstrating a treatment effect in two groups staggered over time and demonstrating the reversal of the treatment effect after the treatment has been removed can provide strong evidence for the efficacy of the treatment. Design can also provide evidence for whether the treatment continues to show effects after it has been withdrawn; Describe factorial designs: Allows for complex experiments with more than one independent variable Including multiple independent variables in the same experiment allows for more sophisticated research questions, and allows the researcher to answer questions about whether the effect of one independent variable depends on the level of another and is referred to as an interaction between the independent variables Each level of one independent variable is combined with each level of the others to produce all possible combinations Each combination becomes a condition of the experiment; Explain between subject, within subject and mixed factorial design: The decision of which approach must be made separately for each independent variable Between subject factorial design, all the independent variables are manipulated between subjects Within subjects factorial design, all of the independent varaibles are manipulated within subjects Mixed factorial design, when you manipulate one independent variable between subjects and another within subjects; Non-manipulated independent variables: In many factorial designs, one of the independent variables is measured but not manipulated Nonmanipulated independent variables are usually participant variables, so they are between-subject factors Studies are generally considered to be experiemnts as long as at least one independent variable is manipulated, regardless of how mnay non manipulated independent variables are included Casual conclusions can only be drawn about the manipulated independent variable; Non-experimental studies with factorial designs: Factorial designs can also include only non-manipulated independent variables, in which case they are no longer experiments but are instead non-experimental in nature; Define interactions: Interaction occurs when the effect of one independent variable depends on the level of another Spreading interactions is when there is an effect of one independent variable at one level of the other independent variable and there is either a weak effect or no effect of that independent variable at the other level of the other independent variable. Cross over interaction is when one independent variable has an effect at both levels of another independent variable, but the effects are in opposite directions; Explain simple effects: A way of breaking down the interaction to figure out precisely what is going on The interaction informs us that the effects of at least one independent variable depends on the level of another independent variable A simple effects analysis allows researchers to determine the effects of each independent variable at each level of the other independent variable; Describe single-subject research: A type of quantitative research that involves studying in detail the behaviour of each of a small number of participants (2-10) Focuses on understanding objective behaviour through experimental manipulation and control, collecting highly structured data, and analyzing those data quantitatively Case studies are quantitative in nature and is the defining difference; What are the assumptions of single subject research: It is important to focus intensively on the behaviour of individual participants It is important to discover casual relationships through the manipulation of an independent variable, the careful measurement of a dependent variable, and the control of extraneous variable; What is the application of single-subject research: Experimental analysis of behaviour, which is used to describe how rewards, punishments, and other external factors affect behaviour over time Applied behaviour analysis is a subfield focused primarily with humans; Reversal design “ABA”: A baseline measurement is established for the dependent variable, the level of responding before any treatment is introduced, and therefore the baseline phase is a type of control condition. Treatment introduced, baseline measured again to determine if it hass changed, treatment removed, baseline measured again to determine if it has changed; Describe multiple-treatment reversal design: Baseline phase followed by a separate phase in which different treatments are introduced; Describe alternating treatments design: Two or more treatments are alternated relatively quickly on a regular schedule; Explain multiple-baseline designs: Issues with reversal design. If a treatment is working, it may be unethical to remove it The dependent variable may not return to baseline when the treatment is removed Multiple baseline designs provide a solution for this in one of three ways, multiple baseline design across participants, behaviours and settings; Hypothesis testing: Goal of most studies is to draw conclusions about a population using a smaller sample Sample statistics are estimations and are influenced by random variability Random variability is referred to as sampling error, which cannot be corrected as it exists in every statistic; Statistical relationship: When there is a statistical relationship in a sample, it is not always clear that there is a statistical relationship in the population A small difference between two groups in the sample may indicate a small difference between two groups in the population, but it could also be that there is no difference between the means in the population and it is just a sampling error Any statistical relationship can be interpreted that there is a relationship in the population and the relationship in the sample reflects this and that there is no relationship in the population, and the relationship in the sample reflects only sampling error; Null hypothesis testing: Formal approach to deciding between two interpretations of a statistical relationship in a sample H0, null hypothesis, no relationship in the population and that the relationship in the sample reflects only sampling error. The sample relationship “occurred by chance” Ha (or H1), alternative hypothesis, there is a relationship in the population and that the relationship in the sample reflects this relationship in the population; Steps for hypothesis testing: Define purpose of the test Define H0 and Ha Set the significance level (a) Calculate the test statistic Determine the degrees of freedom (df) Identify the critical values and rejection region Make a statistical decision Make a clinical decision; Rejecting the null hypothesis: Since the research hypothesis is tested against the null hypothesis, your proposed effect is demonstrated when the null hypothesis is “rejected” If you reject the null hypothesis, you may conclude that the alternate hypothesis is likely to be correct You never “accept” the null hypothesis Absence of evidence is not evidence of absence; Critical values and rejection region: The critical values if the point on the scale of the test statistic beyond which we reject the null hypothesis; Directional hypothesis: There is a specific result that one wants to test so that change must occur in the correct “direction” as compared to the mean. Upper tail and lower tail test, depends on which direction of outcome we are focused in; Non-directional hypothesis: One is comparing change that might occur in either tail of the distribution. This is sometimes called the “two tailed” test There are two critical values and two rejection areas for the null hypothesis; Sample size and strength of relationship: P value requires two considerations, the strength of the relationship and the size of the sample The stronger the sample relationship and the larger the sample, the less likely the result would be if the null hypothesis were true Weak relationships based on medium or small samples are never statistically significant and that strong relationships based on medium or larger samples are always statistically significant; t-tests: Are tests of differences of means One sample t-test is used to compare the mean of a sample to the mean of a population Independent t-test is used to compare the means of two independent groups in order to determine whether there is statistical evidence that the associated population means are significantly different Dependent (or paired) t-test compares the means of two related groups to determine whether there is a statistically significant difference between these means; Assumptions of the t-test: Failure to meet these assumptions can lead to invalid results If one or more of the assumptions cannot be met, then a coservative alpha level should be selected to avoid errors. Assumptions are The population from which the samples are drawn is normally distributed The sample(s) are randomly selected from the population There must be a homogeneity of variance, when drawing two samples they must have approx equal variance, the variance of one group should not be more than twice as large as the other The data must be parametric, meaning on an interval or a ratio scale; Degrees of freedom: Degrees of freedom refers to the number of values in a data set that are free to vary Degrees of freedom for a single data set that is a sample statistic representing a population parameter will ALWAYS be n-1; One tail vs two tailed tests: Two tailed test allots half of the alpha to testing the statistical significance in one direction and half to the other direction. Two tailed tests looks at the probability of the relationship in both directions. One tailed test allots all of the alpha testing the statistical significance in one direction of interest. One tailed tests only look at the probability of the relationship in one direction and ignores the other direction completely; One sample t test: Used to compare sample mean (M) with a hypothetical population mean (uo) that provides some interesting standard of comparison The null hypothesis is that the mean for the population (u) is equal to the hypothetical population mean : u=u0 The alternative hypothesis is that the mean for the population is different from the hypothetical population mean u /=/ u0; Dependent-samples t-test: Used to compare two means for the same sample tested at two different times or under two different conditions. Is appropriate for pretest-posttest designs or within-subjects experiments The null hypothesis is that the means at two times or under the two conditions are the same in the population The alternative hypothesis is that they are not the same; Independent samples t test: Used to compare the means of two seaprate samples (M1 and M2) The two samples might have been tested under different conditions in a between-subjects experiment, or they could be pre-exisitng groups in a cross-sectional design The null hypothesis is that the means of the two populations are the same u1=u2 The alternative hypothesis is that they are not the same u1 /=/ u2; The analysis of variance (ANOVA): T-tests are used to comapre two means When there are more than two groups or conditions t-tests are no longer appropriate One way ANOVA is used to compare the means of more than two samples (M1, M2... MG) in a between subjects design The null hypothesis is that all the means are equal in the population u1 = u2 = uG The alternative hypothesis is that not all the means in the population are equal; Post Hoc testing: Significant F-test indicates that there is a difference in means among the groups in the analysis, it does not indicate which means are different When an independent variable has 3 or more categories, and the F-test is significant, a post hoc test can be done Post hoc test is similar to a t-test, except post hoc tests have a correction for familywise alpha errors built into them A post hoc test is used to locate the means that are significantly different; Bonferroni adjustment: Whenever multiple tests are ran simultaneously, the probability of a significant result increases with each test run The Bonferroni adjustment sets the significance cut of equal The adjustment will overcorrect for type I errors, but can result in an increase in Type II errors and a loss of power; Turkey post-hoc test: Calculates the minimum raw score mean difference that must be attained to declare significance between any two groups Does not permit all possible comparisons, it only permits pairwise comparisons Honestly significant difference, easier to calculate one critical difference and compare all the observed differences to the critical difference; Repeated-measures ANOVA: Differs from one-sample ANOVA in that it has two or more observations on the same subject Is used to compare three or more group means where the participants are the same in each group In repeated measures ANOVAs the subjects serve as their own control; Statistical power: Probability of rejecting the null hypothesis given the sample size and expected relationship strength;