Psychology Research Methodology PDF

5 Research methodology 5 Research methodology Learning outcomes In this chapter, we will discuss:...

5 Research methodology 5 Research methodology Learning outcomes In this chapter, we will discuss: l the six main research methods: experiments, self-reports, observations, case studies, correlations and longitudinal studies l methodological concepts, including aims, hypotheses and variables, experimental design, control of variables and sampling l ethical guidelines for working with humans and non-human animals l ways of evaluating research: validity, reliability and replicability l types of data and data analysis. ▲ Figure 5.1 The bustling city of Jerusalem, the location of the study by Perry et al. Where there are people, there is psychology! Psychology is everywhere, and these people are in the sacred city of Jerusalem, Israel, the nation where Anat Perry and colleagues conducted their research on oxytocin and personal space (see page 143). In this chapter, we draw on examples from Perry et al. (2015) and the other 11 AS core studies as we take an in-depth look at research methods and methodological concepts, the bedrock upon which the science of psychology is built. This chapter provides a whole new language to describe and evaluate psychological research but, before we start, we take a look at how it all began in 1783, with Ferdinand Ueberwasser, a German professor who shared something surprising in common with Dr Anat Perry and with one of the research methods discussed in this chapter. 165 9781398353008.indb 165 08/07/22 11:19 AM  5.0 Introducing research methodology 5 Think! What is a chronoscope and how does it relate to the science of psychology? Think! Imagine you could join the research term of any of the psychologists named in this section, from Ferdinand Ueberwasser to Britta Hölzel, whose team would you join and why? 5 RESEARCH METHODOLOGY Think! Do you think psychologists should spend more time studying thoughts or more time studying behaviour? Psychologists love to observe and Casual talk to people to try to understand observations what makes them think, feel and of everyday life behave in the!ways that they do. Hypotheses When they do this in a more accepted? Theory Asking is supported/ questions systematic and organised way, strengthened they are conducting research!and Hypotheses rejected? Modify collecting data. As you can see, the theory based circle!of science is never-ending; on results there are always more!observations Collecting and analysing data Developing to be made and questions to be asked. and replicating to hypotheses check reliability » Have you observed anything today Designing that made you wonder ‘why’? studies to » You could keep a psychology test hypotheses diary of questions and reflections and your daily ‘whys’. ▲ Figure 5.2 The circle of science Introducing Ferdinand Bernhard Ueberwasser: the grandfather of experimental psychology Socrates, Plato and Aristotle were reflecting on human experience, thoughts and feelings nearly 2,500 years ago but almost 250 years ago Ferdinand Bernhard Ueberwasser appointed himself Professor of Empirical Psychology and Logic at the University of Munster, in Germany. This was the Age of Enlightenment; philosopher Immanuel Kant had coined the phrase Sapere aude (‘Dare to know!’), people were questioning and challenging long-standing beliefs and turning to science for answers. In 1787, Ueberwasser published Instructions, one of psychology’s first textbooks (Ueberwasser, 1787). The book explained how topics such as memory and empathy can be investigated by manipulating variables and replicating observations – as such, it shared a great deal in common with the book you are reading now. Interestingly, Ueberwasser provided a very early account of the placebo effect, which is described in this book on page!178. We opened this chapter with a photo of Jerusalem, as a tribute to the work of Anat Perry, at the University of Haifa, in Israel. She used placebo nasal drops as a control in her scientific study of personal space and role of empathy, showing how the very earliest psychology and the psychology of today share common ground. Sadly, Ueberwasser’s work ended due to the Napoleonic Wars and the closure of his university in 1849. His work has been largely forgotten and Wilhelm Wundt is heralded as the father of experimental psychology, having revived the science of psychology in 1879, in Leipzig. 166 9781398353008.indb 166 08/07/22 11:19 AM  Psychology through the ages In order for psychology to break away from philosophy, as a scientific discipline, it was necessary for researchers to find ways of studying psychological processes, including 5 sensation, perception and feelings, in ways that were objective and reliable. Wundt used an instrument called the Wheatstone-Hipp chronoscope to record reaction times, to within 1/1000 of a second. Unfortunately, his work involved attempts to observe his own private mental processes (a technique called introspection) and this approach was seen, by some, as unscientific. This is because it is impossible to verify what is going on in someone else’s mind; therefore, American psychologist John Watson rejected Wundt’s research into mental events in favour of studying behaviour, which can be observed and confirmed by 5.0 Introducing research methodology multiple researchers. Behaviourists, like Watson, also studied animal behaviour in order to draw inferences about humans. The Cambridge International syllabus includes two animal studies: one contemporary study by Fagen et al., where elephant behaviour is shaped using behaviourist principles; and the second, Hassett et al., which examines the role of sex hormones in shaping monkey toy preferences, and by inference, human preferences. Can you think of one similarity and one difference in the way that data was collected in Fagen et al. versus Hassett et al.? There is one important difference between Fagen et al. and Hassett et al. relating to the research methods discussed in this chapter. Hassett et al. were comparing groups of monkeys (males versus females) and, therefore, their study is classed as experimental, whereas Fagen et al. were not comparing groups or conditions; they were merely observing the elephants to see how long it would take them to learn a set of behaviours. Their study is classed as non- experimental and is a structured observation. Focus on contemporary psychology Despite dominating psychology for nearly 50 years, behaviourism eventually gave way to cognitive psychology in the 1950s. Interest in mental events was renewed and many psychologists agreed that cognitive processes such as memory and attention can be inferred from how people respond in experimental studies. Contemporary cognitive psychologist Joanna Pozzulo also uses laboratory experiments to explore strategies to improve eyewitness testimony, especially in children. This demonstrates one way in which science conducted in the laboratory can have far-reaching implications in the real world, in the pursuit of justice for victims and those who have been falsely convicted. Post-behaviourist psychology not only focused on cognitive processes but also on how our behaviour is influenced by other individuals and groups. Psychology was coming out of the laboratory and into the field. For example, Piliavin et al. (subway Samaritans) staged a mock-emergency on the New York subway in a memorable field experiment. Can you think of one important difference between laboratory experiments and field experiments? Previously, we commented that private mental processes could not be studied empirically (directly through the senses), but advances in technology in the late twentieth century onwards mean that is no longer the case. Techniques such as MRI and fMRI have provided a window into the workings of the brain. Contemporary neuroscientists like Britta Hölzel have used these techniques to study changes in brain structures over time, in experiments with longitudinal designs. One of the strengths of Hölzel et al.’s study was the way it combined objective, reliable, laboratory-based data collection with a real-world Mindfulness-Based Stress Reduction programme where people incorporated mindfulness into their everyday lives, practising at home while washing up or taking a walk. 167 9781398353008.indb 167 08/07/22 11:19 AM  Looking ahead to A Level 5 This research methods chapter covers everything you need to understand and to evaluate the 12 core studies at Cambridge International AS Level and to conduct your own investigations. Planning and conducting research as well as analysing data and reporting your findings are excellent ways to consolidate your growing understanding of methodological concepts (please note, ethical guidelines must be upheld at all times). The chapter also includes a few additional aspects which are relevant to the Cambridge International A Level part of the syllabus only – these are clearly identified by a red line in the margin. We have avoided using examples from the A Level options as different students/schools study different combinations. Whichever options you or your school choose (Clinical Psychology, Consumer 5 RESEARCH METHODOLOGY Psychology, Health Psychology and/or Organisational Psychology), you will learn about additional research methods and methodological concepts, including randomised control trials, postal questionnaires, rating scales: forced/fixed choice, psychometric tests, writing and applying knowledge of null hypotheses and alternative directional (one-tailed) and non- directional (two-tailed) hypotheses, and temporal validity. Before you start… Previously, you learned about the early psychological experiments performed by Wilhelm Wundt. In tribute to his great work on reaction times, we have referred to a mock study by a fictional researcher, Dr Rox, to help you to put your new knowledge into action. Her study is introduced on page!170. You’ll find that she needs your help with her study on caffeine and reaction times quite a lot! Hopefully, you’ll be able to think of some more contemporary ways of measuring reaction times than the chronometer and maybe even find a way to make your measurements both valid and reliable. Good luck! STUDY TIP Although it might be tempting to claim that a certain study proved a certain theory, it is important to avoid the words ‘proof’, ‘prove’ and ‘proved’ in psychology. Philosopher Karl Popper noted that it is impossible to prove the claim that ‘all swans are white’ (Popper, 1959). You could go out searching for swans a hundred times, returning each time with a photo of a white swan, but this does not prove that all swans are white. Popper suggested that tracking down just one black swan would disprove (falsify) the claim – this is why psychologists test null hypotheses (see page 198). This is rather complex, but it is generally better to be cautious when writing in psychology and use phrases like ‘researchers found evidence to support their theory/hypothesis’, or ‘this suggests that…’, or ‘this supports the idea that…’. 5.1 Research methods 5.1.1 Experiments KEY TERMS Think! experiment (laboratory What is the difference between a laboratory experiment and a field experiment? In and field) what way are these types of experiment similar? independent variable dependent variable controlled Think! standardised procedure Imagine you have been asked to conduct a field experiment to investigate obedience at standardised a zoo. What would your independent and dependent variables be? Why might the data instructions you collect be unreliable? laboratory setting natural setting Think! Why might the findings of laboratory experiments lack validity? 168 9781398353008.indb 168 08/07/22 11:19 AM  Experimental group 5 Control group 5.1 Research methods ▲ Figure 5.3 Andrade (doodling) hypothesised that people who doodle while listening to a boring message will recall more of the message than people who do not doodle while listening to the message. She designed an experiment to test her hypothesis What was Andrade’s independent variable (what differed between the two groups)? What was one of her dependent variables (what was measured)? What type of experiment was this? What is an experiment? Think back to the circle of science in Figure!5.2 on page!166 and you will remember that psychologists design and conduct experiments to test hypotheses. They are interested in discovering cause-and-effect relationships between variables. To help reveal the causes of behaviour, thoughts and feelings, they compare how people behave in different situations (conditions), carefully changing one thing at a time (the independent variable) and observing how this affects another thing (the dependent variable). To be sure that changes in the dependent variable really were caused by the independent variable, it is important to think about any other variables that might affect the dependent variable and make sure that they are controlled (kept the same) from one group/condition to the next. A final important feature of experiments is that there is always a carefully standardised procedure. The term procedure means what the experimenter must do in order to collect their data. Standardised means that each step is kept exactly the same for every participant. This includes the use of standardised instructions, the exact same environment (in terms of lighting, room layout, temperature, noise, etc.) and the exact same task. The main features of experiments are as follows: » Experimenters compare data collected from the experimental group or condition with data collected in the control group or control condition in order to draw conclusions about cause and effect. » They have independent variables that are manipulated (changed) and dependent variables that are measured. » Cause and effect can only be inferred if all other variables that could affect the dependent variable are controlled (kept the same for both groups/conditions). » They have a standardised procedure and standardised instructions so that all participants have the exact same experience and so that the study can be replicated to test for reliability. 169 9781398353008.indb 169 08/07/22 11:19 AM  Dr Rox notices that her students always 5 seem more alert following a cup of coffee and decides to carry out an experiment to investigate the effect of caffeine on reaction time. She gives one group of students a 300 ml cup of coffee containing 100 mg of caffeine and tests their reaction time immediately after they have drunk it. She compares their results with another group of students who are 5 RESEARCH METHODOLOGY given a 300 ml glass of water (and no coffee). She predicts that the average reaction time for the caffeine group will IV: the drink (coffee or water) be significantly faster than the average DV: time taken to press the button (milliseconds) reaction time in the no-caffeine group. » The independent variable is ▲ Figure 5.4 Dr Rox’s experiment to whether the participants are given investigate the effect of caffeine on caffeine or water (this is what reaction time. IV, independent variable; Dr Rox manipulated). DV, dependent variable » The dependent variable is participants’ reaction times measured in milliseconds on the reaction time test. When Dr Rox analyses the results, she finds her prediction was correct, but can you think of any reasons why Dr Rox should not conclude that it was the caffeine that caused the faster reaction times? Laboratory experiments Location The term laboratory refers to the setting (location) in which the experiment takes place. A laboratory is any setting over which the experimenter has a high level of control – for example, they can control the temperature, lighting and noise level.!In Dr Rox’s caffeine study, the students would be brought to a specific room at the university where they are given the coffee or water. Next, they would complete a specially designed reaction-time test on a computer – for example, touching the screen as quickly as they can every time they see the number 2 on screen. The time taken to press the button will be measured in milliseconds. Controls Controlled variables are kept the same between all levels (groups or conditions) of the independent variable, to ensure that only the independent variable is causing any effect on the dependent variable. Working in a laboratory setting allows researchers to control many variables. This means the researcher can be much more confident that the independent variable really did cause the changes in the dependent variable. In Dr Rox’s study, if the room was rather hot on the day the coffee group took part but colder on the day the water group took part, the difference in temperature might affect reaction times. Therefore, Dr Rox should control the temperature of the room and the temperature of the water and the coffee. She would also need to control noise levels within the room, ensuring that both groups have the same quiet environment and do not become distracted. Evaluating laboratory experiments Validity Another strength is that, because of the high levels of control, researchers are able to infer cause and effect because they have isolated the variable to be manipulated and the variable that is being measured. For example, in an experiment on the effects of caffeine on reaction time, if the researchers found a significant difference in the results between the caffeine and the control group, they could say that caffeine had an effect on reaction time. 170 9781398353008.indb 170 08/07/22 11:19 AM  A weakness of some laboratory experiments is that they can be low in ecological validity because the tasks participants are asked to perform sometimes do not reflect what they would do in real life. For example, in Pozzulo et al.’s (line-ups) experiment, participants were asked to watch some videos, then identify cartoon and (real) human characters from 5 the videos in photo line-ups. This is not the same as witnessing a crime in real life, where people are unlikely to be expecting the event, may feel more intense emotions and will be viewing the situation in real life not on a screen. These factors are likely to affect their recall for faces in a different way to a calm, controlled setting. This means that the results of laboratory experiments should only be generalised to everyday life with caution as when people are in a real-life setting, they may behave very differently. 5.1 Research methods Another weakness is that validity can also be affected by demand characteristics, which are features of the environment that reveal the aim of the study. If participants are able to work out the aim of the study, they may act in a way they feel the researchers want them to, rather than behaving how they normally would. Reliability One strength of laboratory experiments is that the setting means the procedure can be highly standardised. This means that all participants within each group/condition have the exact same experience as each other. This is important because it means other researchers should be able to replicate the study to test the findings for reliability (i.e. consistency). Ethics A strength is that laboratory experiments should be more ethical than field experiments (see below) because participants know they are taking part in research (even if they have not been told the exact hypothesis). They will have given their consent to take part and will be made aware of their right to withdraw. Unlike field experiments, researchers are generally able to debrief their participants following any deception. This is important as upholding ethical guidelines helps to ensure that participants are treated with respect and that they leave the research setting in the same psychological state as they were in when they arrived. Field experiment Location When psychologists talk about working in ‘the field’, they mean conducting research in real-world settings. Away from the controlled world of the laboratory, field experiments take place in natural environments, where one might expect to see the behaviours of interest, such as the street, supermarket or workplace.!One or more independent variables are still manipulated by the experiment in order to observe the effect on the dependent variable(s). However, the natural environment means that in comparison with laboratory experiments, there may be uncontrolled variables both within and between groups/conditions. These are factors that are not being measured or controlled by the researcher and can have an unwanted effect on the dependent variable. These could be things such as the temperature of the room or noise levels. How could Dr Rox redesign her study as a field experiment? Where would it take place and how could she measure reaction times in a more naturalistic way? Controls When researchers conduct field experiments, they try to maintain control over variables that could affect the dependent variable to allow cause-and-effect conclusions to be drawn, but this is much more difficult than in a laboratory environment. In natural environments, like public places (e.g. Piliavin et al., subway Samaritans), it is not possible to control how many people board the train or how close people are standing to each other, for example. These uncontrolled, situational variables (see page!201) make it hard to draw firm conclusions. 171 9781398353008.indb 171 08/07/22 11:19 AM  Can you think of any variables that would be difficult to control if Dr Rox decided to 5 conduct her study in a coffee shop? Imagine she has ten students in each group, would it be possible to ensure that they all have the exact same experience as each other? Evaluating field experiments Validity A strength of field experiments is that they generally have higher ecological validity than 5 RESEARCH METHODOLOGY a laboratory experiment, meaning that participants’ behaviour is more likely to be natural as they are not in an unfamiliar laboratory setting and often unaware that they are even taking part in the research. This means their results should be generalisable to other similar situations.!Another strength of field experiments is that participants are also much less likely to show demand characteristics if they do not know they are taking part in research. This means that they are unlikely to try to work out what the aim of the research is and act in the way that they think the researchers want them to. Field experiments take place in real-world settings. The behaviours of interest are observed/measured in everyday situations. For this reason, they tend to have higher ecological validity. Also, participants are less likely to be affected by demand characteristics, especially if they do not even know they are in a study. The researcher still manipulates an aspect of the situation (e.g. Piliavin et al. staged an emergency on the subway, see page!154) and so the setting is not completely natural. » Do you think it is possible for a laboratory experiment to have ecological validity? » Do you think it is possible for a field experiment to lack ecological validity? Reliability A weakness is that researchers have much less control in a field experiment. For example, in the Piliavin et al. experiment (see page!153), researchers could not control who was sitting in the train carriage or how crowded it was when the experiment took place. Neither could they control whether the passengers had already seen the confederate collapse in a previous trial, leading them to become suspicious and potentially affecting their response. Field experiments tend to be more difficult to replicate due to the limited ability to standardise the procedure. This makes it difficult to test the reliability of the findings. Ethics A weakness of field experiments is that participants often do not know that they are taking part in research, so consent cannot be sought, which also removes the right to withdraw. When the deception is revealed, it may lead participants to doubt psychologists in the future. Deception should only be used when the research aims cannot be met without it taking place and only if the participants are unlikely to feel discomfort on finding out about the deception. The research design should protect the dignity of the participants. Control groups and/or conditions Experiments involve making comparisons between data obtained in two or more groups or conditions. A strength of control groups/conditions is that the data provided in this part of the experiment provides a baseline to which the data from the experimental group or condition can be compared. Control groups/conditions allow the researchers to confirm that the results of their study are due to the manipulation of the independent variable. In Dr Rox’s study, the water group provides the reaction speed baseline to which the coffee group’s average speed can be compared. Can you think of a different control group that Dr Rox could have used to provide the baseline in her study? 172 9781398353008.indb 172 08/07/22 11:19 AM  LEARNING LINK The study by Bandura et al. (aggression) used a laboratory experiment to investigate 5 whether a child would imitate aggressive behaviour if they witnessed such behaviour in an adult. The children’s behaviour was observed through a one-way mirror while they played with a standardised set of toys that were laid out in exactly the same way for each child, regardless of which group they were in (aggressive model, non-aggressive model or control). The study by Piliavin et al. (subway Samaritans) was a field experiment investigating helping behaviour on a subway train. The subway train was a natural environment for the passengers, who did not know that they were being observed and 5.1 Research methods happened to be riding in the train carriage at the time that the study was being conducted. What similarities and differences are there between these two studies? Think about the independent and dependent variables and how the setting affects the level of control and standardisation; what impact does the setting have on the validity and reliability of the findings of each of these studies? STUDY TIP Sometimes it can be difficult to work out which variable is the independent variable (IV) and which is the dependent variable (DV). There are several ways to think about this. Experimenters generally want to know whether one thing causes an effect on another; they want to know whether the independent variable (I) causes (C) an effect (E) on the dependent variable (D). This spells ICED; you could ask ‘Do sales of iced coffee (the DV) depend on the weather, e.g. hot or cold (the IV)? TEST YOURSELF 1 Outline two features of a laboratory experiment. 2 Explain one similarity and one difference between a laboratory experiment and a field experiment. 3 Andrade used a laboratory experiment to study doodling. Outline one strength of this method with reference to this study. 4 Suggest one advantage of Piliavin et al. (subway Samaritans) conducting their research on a New York subway. 5 Sanija is conducting a study to investigate whether there is a difference in students’ attention spans in school depending on whether they eat a hot or cold breakfast. a Suggest an appropriate research method that Sanija could use to test her theory. b Suggest how Sanija could operationalise her dependent variables. 6 Arthur is conducting an experiment into memory for instructions with a group of children with autism spectrum disorder. He thinks the children will remember the instructions better if they are presented as a written story with pictures compared with reading the instructions aloud. a Identify one variable that Arthur should control in his experiment. b Outline why Arthur should use a standardised procedure in his experiment. 5.1.2 Experimental designs Think! What is the purpose of random allocation? Think! Imagine you are investigating brain localisation and the effect of playing a certain video game. You want to see whether grey matter concentration increases in a certain brain structure following increased time spent playing the game. What research design would you use to investigate this question? How might you improve the validity of your design? 173 9781398353008.indb 173 08/07/22 11:19 AM  Think! 5 How would you rank the three experimental designs in terms of their practicality and validity? Put the experimental designs in order from least to most practical and least to most valid. If possible, discuss this with a classmate to see whether you agree. In Dr Rox’s caffeine experiment (see page!170), the students in the experimental group consumed caffeine and their average reaction time was compared with the average reaction time of the participants in the control group, who did not consume caffeine KEY TERMS (the water group). 5 RESEARCH METHODOLOGY random allocation » Which experimental design was used in this study? experimental design » What were the problems of using this design? experimental group » How could Dr Rox have improved her experiment? control group independent measures baseline Independent measures design repeated measures In an independent measures design, participants are put into two or more different control condition groups, relating to the levels of the independent variable. These groups are sometimes demand characteristics called the experimental group and the control group. Data from the control group provides a order effects (practice baseline to which the researcher can compare the data collected in the experimental group. and fatigue) For example, in the study by Andrade (doodling), participants in the experimental group were asked to shade circles and squares on an A4 piece of paper, while participants in the counterbalancing control group were not allowed to shade or doodle. The average number of words recalled by experimental condition the two different groups of participants was then compared to see if there was a difference matched pairs between them. An independent measures design uses different participants for each condition of the experiment. In what way might using different participants for different conditions affect the validity of the results, and how might this be avoided? Random allocation and participant variables Another key feature of laboratory experiments is that the researchers are able to randomly allocate participants into either the experimental or the control group. This is often done by assigning each participant a number and then using a random number generator to select participants for the two or more groups. Without random allocation, it is possible that members of one group all share some common characteristic that makes them different from the people in the other group. These characteristics are called participant variables, and may be things like gender, age or level of intelligence or empathy. When this happens, it is impossible to know whether differences in the dependent variable result from the manipulation of the independent variable or the participant variables. The purpose of random allocation, therefore, is to increase validity (see below). Dr Rox realises that there might be some from the drink consumed before the test or important participant variables affecting differences in the students’ regular caffeine the students’ results in her study on the consumption. How could Dr Rox reduce the effects of caffeine on reaction times effect of participant variables in her study? (see page!170). An important participant variable in her study might be how much coffee (and other caffeinated drinks/ snacks) the students usually consume. If all the heavy-caffeine users end up in the coffee group and all the light-caffeine users, by chance, end up in the water group, it would be unclear whether any difference in average reaction times results ▲ Figure 5.5 174 9781398353008.indb 174 08/07/22 11:19 AM  Repeated measures 5 5.1 Research methods ▲ Figure 5.6 In repeated measures designs, the same participants take part in both the experimental condition and the control condition In a repeated measures design, the same group of participants provide data relating to both levels of the independent variable; they act as their own control group. This means participant variables cannot affect the validity of the data as every participant takes part in both the experimental and the control conditions. The control condition provides a baseline for comparison and allows the researcher to see whether the independent variable has caused the dependent variable to increase or decrease. Unfortunately, the repeated measures design has problems of its own regarding validity: demand characteristics and order effects. Demand characteristics Demand characteristics are cues in the set-up of the experiment, including how the researcher speaks and behaves, that provide clues about the aims and/or hypotheses of the study. Participants sometimes, consciously or unconsciously, change their behaviour in response to these cues and this reduces the spontaneity of their behaviour, meaning they may not behave as they usually would. This may be because they want to please the experimenter or behave in ways that they think are expected of them. Demand characteristics are, therefore, a threat to the validity of the study’s findings and action should be taken to minimise them where possible. Order effects, including fatigue and practice effects When participants have to perform a task twice under different conditions, behaviour in the second condition may be different to the first, not because of the changes in the independent variable, but due to the fact they have performed the task once already. For example, they may perform better the second time due to the experience gained in the first condition (a practice effect) or increased motivation, or worse due to boredom, tiredness or the strain of concentrating in the first condition (a fatigue effect). Therefore, order effects are a threat to validity as it becomes unclear whether the independent variable is affecting the dependent variable or whether the order of the conditions has brought about any changes observed. One way researchers can reduce the impact of order effects is to leave a significant time period between first and second conditions. When this is not possible, they can be addressed using a technique like counterbalancing. 175 9781398353008.indb 175 08/07/22 11:19 AM  In Dr Rox’s experiment on the effects of second condition as they have already 5 caffeine on reaction time (see page!170), the participants might perform better the been concentrating on the screen in the first condition (fatigue effect). They might second time they do the reaction time test also need the toilet if they have consumed because they have already done it once both a cup of coffee and a glass of water! and had the chance to practise. Similarly, How could you set up the experiment to participants may perform worse in the avoid order effects? Counterbalancing 5 RESEARCH METHODOLOGY Counterbalancing is a technique that can be used in a repeated measure design so that order effects do not compromise the validity of the findings. Participants are put into one of two groups: group one does the experimental condition first and then the control condition, and group two does the control condition first and then the experimental condition. This way, although practice effects will still occur, their impact on the findings is reduced. Matched pairs design In a matched pairs design, each participant in the experimental group will be matched with a participant in the control group on a number of relevant factors. This means for every participant in the experimental group, the control group will contain a person who is very similar to them on key variables that might have otherwise affected the dependent variable of the study. For example, participants could be matched on handedness (preference for using the right or left hand), gender, socioeconomic status or age. A matched pairs design overcomes the problem of participants’ variables and order effects, but in practice, they can be difficult to set up as careful screening of the participants is necessary in order to match the groups. It can also be difficult to recruit people who have the necessary characteristics. Dr Rox is still concerned that participant Experimental Control group: variables might affect the validity of her group: Elina, female, Soraya, female, aged 22, conclusions about the effect of caffeine on aged 22, left-handed reaction times. She decides to use a matched left-handed pairs design where she will match the participants in each condition on age, sex, handedness and experience in playing video games, all variables which may affect the validity of her conclusions. What would she need to do to set up her matched groups of participants?  Figure 5.7 In matched pairs designs, participants are matched across the experimental and control conditions LEARNING LINK The research by Hölzel et al. (mindfulness and brain scans) uses a combination of both independent measures design and repeated measures design (see page 29). Participants in the experimental group took part in an eight-week Mindfulness-Based Stress Reduction (MBSR) programme. MRI brain scans were conducted both before and after the intervention. Data from the scans was compared to see if the MBSR programme had changed grey matter concentration. Which part of Hölzel’s study is independent measures and which part is repeated measures? Why was it important in this study to also have a control group? 176 9781398353008.indb 176 08/07/22 11:19 AM  STUDY TIP Once you are confident in being able to explain the strengths and weaknesses of the different experimental designs, ensure that you are able to apply this knowledge and 5 understanding to any new psychology experiments that you learn about in class or read about online. Working with a partner, if you can, why not design a set of experiments for each other on topics like sleep, memory, aggression and helpfulness? Write out the independent and dependent variables you would like to investigate. Roll a dice to see whether the design will be independent measures (odd numbers) or repeated measures (even numbers). Write a sentence on how you would set your study up and then swap with your partner and identify the problems in each study (e.g. specific participant variables, how order effects might alter 5.1 Research methods the results) and re-design the studies to overcome the problems. Being able to evaluate the experimental design, using key features from the study, is an essential skill. TEST YOURSELF 1 Outline what is meant by ‘counterbalancing’ using any example. 2 Outline one strength and one weakness of a matched pairs design. 3 Pozzulo et al. used a repeated measures design so children observed both human and cartoon faces and they also participated in both target-present and target- absent line-ups. Suggest one advantage of a repeated measures design with reference to this study. 4 In Piliavin et al. (subway Samaritans), the researchers observed train passengers’ reactions to a staged emergency. The design was independent measures. Explain one disadvantage of this design using this study as an example. 5 Javine thinks ink colour (red or black) might affect the speed at which people are able to solve anagrams. She decides to conduct a laboratory experiment with a repeated measures design to test this idea. a Explain how practice effects could affect the validity of Javine’s findings. b Explain how Javine could use counterbalancing to improve her study. 6 Fabian shows his participants a painting. He tells one group that it took the artist six months to complete and another group was told it took six years to complete. He asks the participants to rate how much they liked it on a scale from 1–10. a Identify the experimental design used in Fabian’s experiment. b Explain one disadvantage of using this experimental design in this study. 5.1.3 Randomised control trials KEY TERMS Think! placebo effect What is meant by a placebo effect? How are placebo effects controlled in a randomised randomised control trial control trial (RCT)? placebo expectancy effects Think! single-blind Your friend says she cannot concentrate in her study periods unless she has had a can double-blind of her favourite brand of cola first. How could you design a study using a RCT to see whether your friend’s favourite brand really improves her concentration or whether she just thinks that it does? Can you think of any problems with your study? Think! RCTs sometimes raise ethical issues as when participants are assigned to a placebo/ control group, this may mean they have to wait for a treatment that could improve their quality of life. Can you think why such studies may be ethically justifiable? 177 9781398353008.indb 177 08/07/22 11:19 AM  One special type of experiment is called a randomised control trial (RCT). Randomised control trials (RCTs) are a special type of experiment. They use various techniques to 5 increase validity and they are often referred to as the ‘gold standard’ in clinical and health research, such as drug testing. For example, participants might be randomly assigned to either an experimental group (who are given the new drug) or a control group. The control groups in RCTs sometimes receive no treatment or they receive a placebo. The placebo effect Sometimes it is hard to know whether the positive effect of a new drug is due to the drug 5 RESEARCH METHODOLOGY itself or the fact that participants expect their symptoms to improve once they begin the new course of treatment. In order to isolate how much of any improvement is due to the drug and how much is due to expectancy effects, researchers often use placebo treatments. Here participants in the control group receive a treatment that is administered in exactly the same way as the real treatments (e.g. in the case of orally administered tablets, the tablet will look, smell and taste just like the real drug) but contains no active ingredients. Importantly, the participants do not know whether they have received the real drug or the placebo. This is called a single-blind design. Changes in symptoms (before and after treatment) are calculated for both groups. For example, in the study by Perry et al. (personal space), participants were asked to self-administer either oxytocin or a saline solution (placebo), before taking part in experiments on personal space. They were unaware whether they had received the drug or the placebo (the saline solution) to avoid this knowledge affecting their behaviour. Often, placebo groups experience significant improvements over time (the placebo effect) but any significant difference in the amount of improvement between the experimental and control groups must be caused by the independent variable – that is, the presence of the active ingredient in the real drug. RCTs are also used in studies investigating the efficacy of psychological treatments such as cognitive behavioural therapy (CBT) and other interventions to improve health and wellbeing. Here the placebo treatment is something that is similar to the genuine treatment/intervention in many ways (e.g. weekly appointments with a counsellor), but without the CBT element. Double-blind design As noted above, participants do not know whether they have received the placebo or the real drug but often the researcher who measures the severity of the participants’ symptoms is also unaware whether the participants are in the placebo group or treatment group. This is called a double-blind design. In some studies, the person analysing the data is also unaware whether data points were contributed by people in the placebo or intervention group and this is known as a triple-blind design! Evaluating RCTs Validity A strength of the double-blind design is that it reduces experimenter bias as well as reducing demand characteristics. For example, if the experimenter knew who was in the placebo group they might expect those participants to have experienced less improvement over the course of the study. The experimenter’s expectations could affect the data provided by the participant, reducing the validity of the findings. A double-blind design makes sure this cannot happen, ensuring the experimenter’s assessment of the participant remains objective. Reliability Another strength is that the double-blind design allows researchers to test interventions (e.g. treatments and therapies) in a controlled way. As none of the experimenters who come into contact with the participants are aware which groups they are in, all participants are treated in exactly the same way by the experimenters, meaning that their assessments can be replicated and tested for inter-rater reliability. 178 9781398353008.indb 178 08/07/22 11:19 AM  Ethics A weakness is that valid, informed consent is not possible. Double-blind RCTs rely not only on the researchers being unaware which group (placebo or genuine treatment) the 5 participant is in, but also the participant, meaning they are not giving fully informed consent. This is important, however, as it preserves the validity of the findings and arguably the benefits of the research outweigh the cost to participants, especially given that the participants have consented to the possibility that they will receive a placebo. A weakness of RCTs is that it could mean that participants randomly assigned to the control group do not have access to treatments that could significantly improve their quality of 5.1 Research methods life. In order to still conduct important RCTs on treatments for mental health disorders, the control groups are often offered the treatment they missed out on as soon as the study has ended. The start of treatment, however, would still be delayed, potentially causing unnecessary psychological harm. They are often referred to as a wait-list control group. In Dr Rox’s study about the effect of caffeine on reaction times (see page!170), the control group drank a cup of water. Now that you have a better understanding of expectancy effects, you should be able to explain why the coffee group might have experienced faster reaction times than the water group for psychological rather than biological reasons. » How could Dr Rox have turned her study into a RCT with a placebo group and a double-blind design? LEARNING LINK Lewin et al. (1992) used a randomised control trial to test the effectiveness of a home- based exercise programme to help improve fitness after a heart attack (see page 617). Participants were randomly assigned to receive either the self-help rehabilitation intervention or standard care plus a placebo package of information and informal counselling. TEST YOURSELF 1 Explain one strength and one weakness of using randomised control trials. 2 Suggest one ethical issue which may arise from the use of randomised control trials, using any example. 3 Dr Phil would like to test a new drug he has developed to support individuals with gambling disorders. Suggest how Dr Phil might use a randomised control trial to test whether the new drug is more effective than a placebo. 4 Hajra would like to find out whether yoga has a positive impact on people’s subjective wellbeing, as previous research has suggested. Suggest one reason why a randomised control trial may not be a suitable method to investigate this. 5 Malcolm is investigating the effectiveness of electro-convulsive therapy (ECT) as a treatment for schizophrenia. He plans to use a randomised control trial where the control group think they have been given ECT but have not. a Suggest two ways that Malcolm can ensure that his data is objective. b Outline one difference and one similarity between the experimental and control groups in Malcolm’s study. 6 Jolene has had pain in her shoulder for months following a car accident. She is participating in a randomised control trial to test the effectiveness of acupuncture. Explain one ethical issue relating to randomised control trials. You must refer to Jolene in your answer. 179 9781398353008.indb 179 08/07/22 11:19 AM  5.1.4 Self-reports: questionnaires 5 KEY TERMS Think! open question What is the difference between an open question and a closed question? closed question self-report Think! questionnaire interview Imagine you are investigating the effect of stress on the content of people’s dreams. Which type of question (open or closed) would be best to find out about people’s stress 5 RESEARCH METHODOLOGY rich (as in data) levels and their dream content? target population Think! Why do you think some researchers prefer rating scales with an even number of choices, such as six-point scales from 0–5, and others prefer to use scales with an odd number, such as seven-point scales from 0–6? Self-reports are a method of research where the participant responds to either open or closed questions. Data may be gathered about their beliefs and opinions as well how they believe they would behave in certain situations. Self-reports include both questionnaires and interviews and may collect both quantitative and qualitative data. Questionnaires Questionnaires can be completed in a number of ways. Printed questionnaires may be completed by hand, but nowadays most questionnaires are distributed digitally and participants complete them online. This saves printing and postage costs and data can be collated much quickly and easily. This said, online questionnaires can only be completed by participants who have access to the internet and understand the technology, meaning traditional ‘pen and paper’ surveys are still used and may be more convenient for many types of research. Closed questions Closed questions offer a fixed choice of answers. They may use some form of scale such as a Likert scale, in which the participant can circle or tick the response to show the extent to which they agree with a statement. An example of a Likert scale might be: I do not like people invading my personal space, especially if I do not know them. 1 2 3 4 5 Please circle the response that best fits your view: 1 = Completely disagree, 2 = Disagree, 3 = Neither agree nor disagree, 4 = Agree, 5 = Completely agree. Here is another example of a closed question is: ‘Do you find mindfulness meditation relaxing? Yes/No.’ This type of closed question may not provide enough options to reflect the participants’ range of experiences with mindfulness meditation. For example, it is not possible to communicate that you occasionally find mindfulness meditation relaxing but often find it ineffective. In this example, a rating scale may allow participants to provide more valid responses. For example, To what extent do you agree: Mindfulness meditation helps me to relax 0 = Never found it helpful to 5 = Aways very helpful. Although dichotomous questions (with only two answers) are often not very useful, they may be helpful if you wish to provide contingency questions. These allow the participant to ignore questions that are not relevant to them. For example: Have you ever tried mindfulness meditation as a way of relaxing? Yes/No. If yes, please answer questions 2–5. If no, please continue to question 6. 180 9781398353008.indb 180 08/07/22 11:19 AM  Rating scale: forced/fixed choice Some of the studies in the options topics use rating scales where respondents must make a forced/fixed choice. This type of scale does not allow the participants to respond with 5 answers like ‘unsure/undecided/don’t know/neutral’ as it is intended to ‘force’ the participant to make a response and express an opinion. Forced-choice questions are often written as a statement that participants have to agree or disagree with or give a ‘yes/no’ answer to. Fixed-choice questionnaires produce quantitative data, which allows for objective comparisons to be made between people; however, participants’ answers may lack validity as the fixed responses may not exactly match what they would like to say. 5.1 Research methods Open questions Open questions allow the participants to express their opinion more freely and tend to start with words such as ‘why’ and ‘how’. These words encourage participants to give detailed, qualitative answers. The data they provide is often described as rich because it allows the researcher to understand the complexity of the issues they are asking about. This is because participants are able to explore the reasons behind their initial thoughts and feelings in greater depth. These sorts of questions also allow the researcher to understand the diversity of an issue as the answers provided by the participants may be very different from one another. ▼ Table 5.1 Evaluating open and closed questions Closed questions Open questions Advantages Data is quantitative (answers can be categorised Data is qualitative, so it is rich and detailed, giving and counted) and can be analysed using measures researchers deeper insight than would be possible of central tendency and/or spread and displayed with quantitative, fixed-choice answers, visually using graphs and charts. Forced/fixed-choice rating scales do not allow participants to avoid answering questions. Disadvantages Limited choice reduces validity. A fixed range of Data can be harder to analyse than for closed answers mean participants cannot freely express their questions. As there may be a huge amount of data, opinions; their views may not fit with the fixed-choice analysis can be time-consuming. It can also be answers available. subjective as researchers summarise and report on key themes in the data; other researchers might interpret the data differently, making findings less reliable than quantitative data from closed questions. STUDY TIP When analysing data from a questionnaire, it is useful to remember that yes/no answers can be summarised by using the mode as a measure of central tendency. If you have a Likert scale, where people indicate their agreement on a numerical scale, you can use the mode or the median. If you want to try creating an online questionnaire, there are many free programs you can use to do so. What to avoid when creating questionnaires 1 Double-barrelled questions: these are questions where two things are asked in one question, such as: I think that students should have more exams and coursework. 1 2 3 4 5 Please circle the response which best fits your view: 1 = Completely disagree, 2 = Disagree, 3 = Neither agree nor disagree, 4 = Agree, 5 = Completely agree. It might be that the participant thinks that more coursework would be beneficial but does not want more exams. However, it is not possible for them to make this clear in their answer. 181 9781398353008.indb 181 08/07/22 11:19 AM  2 Ambiguous (unclear) questions: it is extremely important that participants can understand what the question is actually asking. If not, it will affect the validity of their 5 responses. Pilot studies are useful to ‘trial’ drafts of newly developed questionnaires. 3 Technical terms: avoid using technical language unless your target population will understand it. For example, if you were only going to be using doctors as your target population, it would be acceptable to use medical terminology, but not if your target population had no medical training. 4 Emotive language: you should avoid using words that are likely to offend or upset your participants for ethical reasons but also emotive language may be persuasive and lead participants to give an answer that they would not have given if the question was worded differently. 5 RESEARCH METHODOLOGY STUDY TIP If you are creating a questionnaire, it is best to undertake a small pilot study on a few participants before distributing it to a larger number of people. This will help you to ensure that participants understand all of the questions and are able to answer as many as possible. If the pilot study highlights any problems, these can be resolved before collecting any further data. Evaluating questionnaires ▼ Table 5.2 Advantages and disadvantages of questionnaires Advantages Disadvantages Relatively quick and easy to obtain the views Participants may be less willing to elaborate of a large number of participants; large fully on answers to open questions when sample size means results should be more writing, therefore limiting the depth of data representative, although this also depends on collected. the sampling technique (see page!199). Participants may be less likely to give socially Participants may still give socially desirable desirable answers than during interviews answers. These are answers that make because they are not answering face-to- them look good or acceptable within their face and their responses can be provided society but do not actually reflect their own anonymously. viewpoint. STUDY TIP Although researcher bias and subjectivity can be an issue in the analysis of qualitative data, it does not mean that all researchers will show researcher bias. When evaluating methods and studies, it is best to use words such as ‘could’ and ‘might’ to show that bias is a possibility rather than a certainty. Psychologist Hans Eysenck created a well-known personality test called the Eysenck Personality Inventory (EPI). It includes nine questions to assess whether a participant is likely to have given socially desirable answers or not (the lie scale) (Eysenck and Eysenck, 1991). Anyone scoring five or above was probably lying. This allows researchers to tell if the participant is likely to be giving truthful answers about the rest of the personality questions. For example, one question on the EPI is ‘If you say you will do something do you always keep your promise, no matter how inconvenient it might be to do so?’ If the participant responds ‘yes’ to this question, it is likely that they are lying and giving a socially desirable answer as most people at some point break a promise.!Do you think it is valid to assume that participants lied on other questions based on their answers to the lie scale questions? 182 9781398353008.indb 182 08/07/22 11:19 AM  Postal questionnaires Postal questionnaires refer to questionnaires that are posted out to the sample.!For example, Lewin et al. (1992) used postal questionnaires to assess the psychological impact 5 of a rehabilitation programme for people who had had a heart attack (see page!617). The questionnaires measured anxiety and depression, general health and use of the health services. ▼ Table 5.3 Evaluating postal questionnaires Advantages Disadvantages 5.1 Research methods Relatively quick and easy to obtain the The return rate for postal questionnaires is quite views of a large number of participants; low due to the effort of posting them back; large sample size means results should participants may not have the motivation to do be more representative, although this this. Participants who do return the questionnaires also depends on the sampling technique may be unrepresentative of the target population!– (see page!205). for example, they may be especially helpful, motivated or have a particular interest in the topic being studied. Can be posted anywhere in the world; It can be expensive to post large numbers of location is not an issue as it would be questionnaires, especially if return postage is with a face-to-face interview. included. LEARNING LINK Questionnaires can also be used to collect data as part of an experiment. For example, Pozzulo et al. (line-ups) used an eight-item questionnaire called the Demographic and Cartoon Watching Form to gather information from the parents of the children in the study (see page 70). This information included age, gender, primary language and ethnicity, whether they had any siblings and how much time they spent watching cartoons. Questionnaires like this can be helpful in providing key information that is required when making generalisations from the sample to the target population. Also, in experiments with matched-pairs designs, questionnaires may be useful to gather data to match the participants, as part of the initial screening. TEST YOURSELF 1 Suggest one advantage of asking open questions in questionnaires. 2 Suggest one disadvantage of using online questionnaires. 3 Baron Cohen et al.’s Eyes Test is a ‘pen and paper’ test. Suggest one strength of using this format. 4 Pozzulo et al. (line-ups) used a questionnaire to find out more about the children in their study. State one closed question that they could have asked about children’s cartoon watching. 5 Lucca works with elephant handlers (mahouts) in Nepal. He is using a questionnaire to investigate the handlers’ attitudes towards training methods that focus on positive reinforcement. a Explain how one ethical guideline would be important for Lucca’s study. b Suggest two advantages of Lucca using a questionnaire to gather data for his study. 6 Carol creates a questionnaire including a series of fixed/forced-choice questions to measure helping behaviour in a variety of different situations. Identify two questions that Carol could include in her questionnaire. 183 9781398353008.indb 183 08/07/22 11:19 AM  5.1.5 Self-reports: interviews 5 KEY TERMS Think! structured interview What are the key features of a semi-structured interview? unstructured interview semi-structured Think! interview in-depth Imagine you are investigating unusual phobias (e.g. buttons, flowers). You are interested to know whether people can remember any traumatic events that might have become 5 RESEARCH METHODOLOGY associated with their feared object. Why might an interview be more effective than a questionnaire for collecting data about this topic? Think! There are many factors that can affect the validity of the data obtained in an interview, from the experience and appearance of the interviewer to the location of the interview. What advice might you give to a researcher to ensure their interview data is valid?