AQA Psychology A-Level Past Paper PDF

Summary

These are detailed notes for AQA Psychology A-level, covering Research Methods. The document explains experimental methods, aims, hypotheses, and variables (independent and dependent).

Full Transcript

AQA Psychology A-level Topic 7: Research Methods Detailed Notes This work by PMT Education is licensed under https://bit.ly/pmt-cc https://bit.ly/pmt-edu-cc CC BY-NC-ND 4.0...

AQA Psychology A-level Topic 7: Research Methods Detailed Notes This work by PMT Education is licensed under https://bit.ly/pmt-cc https://bit.ly/pmt-edu-cc CC BY-NC-ND 4.0 https://bit.ly/pmt-cc https://bit.ly/pmt-edu https://bit.ly/pmt-cc Experimental Method The experimental method concerns the manipulation of an independent variable (IV) to have an effect on the dependent variable (DV), which is measured and stated in results. These experiments can be: field, laboratory, quasi or natural. Aims An aim is a general statement made by the researcher which tells us what they plan on investigating, the purpose of their study. Aims are developed from theories and develop from reading about other similar research. Hypotheses A hypothesis is a precise statement which clearly states the relationship between the variables being investigated. The hypothesis can either be non-directional or directional. A directional hypothesis states the direction of the relationship that will be shown between the variables whilst a non-directional hypothesis does not. E.g. If a researcher is carrying out a study to investigate whether sleep helps memory performance: A directional hypothesis for this would be - “The more sleep a participant has the better their memory performance.” A non-directional hypothesis would be - “The difference in the amount of hours of sleep a participant has will have an effect on their memory performance, which will be shown by the difference in the memory test scores of the participants.” A directional hypothesis tends to be used when there has already been a range of research carried out which relates to the aim of the researcher’s investigation. The data from this previous research would suggest a particular outcome. However if there has been no previous research carried out which relates to the study’s aim or the research is contradictory than a non-directional hypothesis is appropriate. Independent and dependent variables The independent variable refers to the aspect of the experiment which has been manipulated by the researcher or simply changes naturally to have an effect on the DV which is then measured. The dependent variable is the aspect of the study which is measured by the researcher and has been caused by a change to the IV. All other variables that could affect the DV should be carefully controlled so that the researcher is able to confidently conclude that the effect on the DV was caused by only the IV. In order to properly test the effect of the IV we need different conditions: the experimental condition and the control condition. You can have various experimental conditions which will allow you to compare the effects of different levels of the IV. Operationalisation of variables Operationalisation refers to the act of a researcher clearly defining the variables in terms of how they are being measured. This means the variables should be defined and measurable. The hypotheses states should also show this operationalisation e.g. the aforementioned directional hypothesis would be even better if operationalised: https://bit.ly/pmt-cc https://bit.ly/pmt-edu https://bit.ly/pmt-cc “Participants that get at least four hours of sleep will show better performances on the memory test, shown by them achieving higher scores than the participants that got less than four hours of sleep.” It could even be further operationalised when more details of the investigation are given, such as the number of questions in the test, hence the maximum score a participant can achieve. Control of Variables Extraneous variables and confounding variables In an experiment, the only aspect that should affect the DV is the IV. Any other variables that may interfere with the IV or the DV should be removed from the experiment or well controlled. Such variables can be confounding or extraneous. An extraneous variable refers to any other variable which is not the IV that affects the DV and does not vary systematically with the IV, they are essentially nuisance variables. Examples are the lighting in the lab or the age of participants - these variables do not confound the results of a study but just make them harder to detect. A confounding variable is also described as a variable other than the IV which has an effect on the DV. Unlike the extraneous variable, confounding variables do change systematically with the IV. With these variables it becomes difficult for the researcher to be sure of the origin of the impact of the DV as the confounding variable (not the IV) could have been the cause. An example for the aforementioned sleep study would be time of day the experimental task is done - those who complete the memory test later in the day may be more tired and therefore do worse, obscuring the true relationship between lack of sleep and memory performance. Therefore, potential confounding variables must be identified and controlled; in this case the participants should take the test at the same time of day. Demand characteristics and Investigator effects Demand characteristics refer to any cue the researcher or the research situation may give which makes the participant feel like they can guess the aim of the investigation. This can cause the participant to act differently within the research situation from how they would usually act. This is as participants from the start of the experiment are trying to figure out what's going on in this new situation they find themselves in - this is known as participant reactivity. They may change their behaviour to fit the situation rather than acting naturally. They may act in a way they think the researcher wants them to which is known as the ‘Please-U effect’ or they may intentionally underperform to sabotage the study’s results, the ‘screw-U effect’. This unnatural behaviour then affects the validity of the results, hence demand characteristics provides a problem for research. Participant reactivity may also lead to investigator effects which refers to any unwanted influence from the researcher’s behaviour, either conscious or unconscious, on the DV measured (the research’s results). This includes a variety of factors :- the design of the study, the selection of participants and the interaction with each participant during the research investigation. Randomisation and Standardisation To minimise the effects of extraneous or confounding variables different steps can be taken by the researcher like randomisation and standardisation. Randomisation is the use of chance to https://bit.ly/pmt-cc https://bit.ly/pmt-edu https://bit.ly/pmt-cc reduce the effects of bias from investigator effects. This can be done for the design of materials, deciding the order of conditions, the selection of participants e.t.c. Standardisation describes using the exact same formalised procedures and instructions for every single participant involved in the research process. This allows there to eliminate non-standardised instructions as being possible extraneous variables. Experimental Method: Types of Experiment Design Description Strengths Limitations Laboratory An experiment that High degree of control- Experimenter’s bias- this takes place in a experimenters control all bias can affect results and special environment variables,the IV has been participants may be whereby different precisely replicated, influenced by these variables can be leading to greater expectations. carefully controlled. accuracy. Low ecological validity- Replication - researchers high degree of control can repeat experiments makes the situation artificial, and check results. unlike real life. Field An experiment Naturalistic - so more Ethical considerations- conducted in a more natural behaviours hence invasion of privacy and natural environment, high ecological validity. likely to have been no not in a lab but with Controlled IV informed consent. variables still being Loss of control- over well controlled. extraneous variables hence precise replication not possible. Quasi An experiment Controlled conditions- Cannot randomly allocate whereby the IV has hance replicable, likely to participants- to conditions not been determined have high internal validity. so there may be by the researcher, confounding variables instead it naturally presented. This makes it exists e.g gender harder to conclude that the difference studies. IV caused the DV. Natural An experiment in Provides opportunities- Natural occurring events- which the IV is not for research that would may be rare this means brought about by the have otherwise been these experiments are not researcher hence impossible due to likely to be replicable hence would have happened practical or ethical hard to generalise findings. even if the researcher reasons. Very difficult to had not been there High external validity- as randomise- participants e.g. if studying you are dealing with real into groups so confounding reactions to life issues. & extraneous variables earthquakes. become a problem. https://bit.ly/pmt-cc https://bit.ly/pmt-edu https://bit.ly/pmt-cc Sampling The researcher needs to decide how they select participants to take part in their investigation.The population is a group of people from whom the sample is drawn. E.g. If the sample of participants are taken from the sixth formers going to schools in London, the findings of the study can only be applied for that certain group of people and not all the sixth formers in the UK. There are various methods that a researcher can use to select participants: Sampling Method Explanation Strengths Limitations Opportunity Participants happen to Easy method of Not representative of sampling be available at the time recruitment which is the whole population which the study is being time saving and less hence lacks carried out so are costly. generalisability. recruited Researcher bias is conveniently. presented as they control who they want to select. Random sampling This is when all No researcher bias- Time consuming- members of the researcher has no need to have a list of population have the influence of who is members of the same equal chances picked. population (sampling of being the one that frame) and then is selected. The contacting them takes method used is :- each time. member of the Volunteer bias- population is assigned participants can a number then either a refuse to take part so random number table can end up with an or a random number unrepresentative generator or the sample. lottery method is used to randomly choose a partner. Systematic A predetermined Avoids researcher Not truly unbiased sampling system is used bias and usually unless you use a whereby every nth fairly representative random number member is selected of population. generator and then from the sampling start the systematic frame. This numerical sample. selection is applied consistently. https://bit.ly/pmt-cc https://bit.ly/pmt-edu https://bit.ly/pmt-cc Stratified sampling With this method the No researcher bias- Time consuming to composition of the the selection within identify strata and sample reflects the each stratum is done contact people from varying proportions randomly. each. of people in particular Produces A complete subgroups (strata) representative data representation of the within the wider due to the target population is population. Firstly you proportional strata not possible as the identify strat. Then you hence generalisation identified strata calculate the required is possible. cannot reflect all the proportion needed for differences between each stratum based on the people of the the target population. wider population. Then select sample at random from each stratum using a random selection method. Volunteer sampling Involves self selection Quick access to Volunteer bias- they whereby the participant willing participants study may attract a offers to take part either which makes it easy particular profile of a in response to an and not time person. This means advert or when asked consuming. generalisability is then to. As participants are affected. willing to take part Motivations like they are more likely money could be to cooperate in the driving participation so study. participants may not take study seriously, influencing the results. https://bit.ly/pmt-cc https://bit.ly/pmt-edu https://bit.ly/pmt-cc Experimental Design Design Description Strengths Limitations Solution Independent The participants -There are no - No control over Random groups only perform in order effects participant allocation solves design one condition of presented. variables the first limitation the independent - Participants are whereby different mentioned.This is variable (IV). less likely to abilities of as it ensures that guess the aims of participants in the each participant the study various conditions has the same (demand can cause chance of being in characteristics changes to the one condition of are eliminated). DV.. the IV as another. -You need more participants than other designs to gather the same amount of data. Repeated The same - Eliminates - Order effects Counterbalancing measures participants take participant presented e.g. - this is when half part in all variables. boredom may of the participants conditions of the - Fewer mean in second do conditions in IV. participants condition done one order and the needed, so not participant does other half do it in as time not do as well on an opposite order. consuming task. finding and using them. Matched Pairs of - No order - Time pairs participants are effects. consuming and first matched on - Demand expensive to some variable characteristics match that has been are less of a participants. found to affect the problem. - A large pool of dependent potential variable (DV), participants is then one member needed which of each pair does can be hard to one condition get. and the other - Difficult to know does another. which variables are appropriate for the participants to be matched. https://bit.ly/pmt-cc https://bit.ly/pmt-edu https://bit.ly/pmt-cc Pilot Studies A pilot study is a small-scale version of an investigation which is done before the real investigation is undertaken. They are carried out to allow potential problems of the study to be identified and the procedure to be modified to deal with these. This also allows money and time to be saved in the long run. Single-Blind and Double-Blind Procedures Single-blind procedure A research method in which the researchers do not tell the participants if they are being given a test treatment or a control treatment. This is done in order to ensure that participants do not bias the results by acting in ways they “think” they should act-avoids demand characteristics. Double-blind procedure A research procedure in which neither the participants nor the experimenter knows who is receiving a particular treatment. This procedure is utilised to prevent bias in research results. Double blind studies are particularly useful for preventing bias due to demand characteristics or the placebo effect. Gives a way to reduce the investigator effects as the investigator is unable to unconsciously give participants clues as to which condition they are in. Control group/condition - sets a baseline whereby results from the experimental condition can be compared to results from this one. If there is a significantly greater change in the experimental group compared to the control than the researcher is able to conclude that the cause of effect was the IV. Observational Techniques Type of observation and Strengths Limitations description Naturalistic- watching and - High ecological validity - Low ecological validity if recording behaviour in the -High external validity as participants become aware setting where it would done in a natural environment that the are being watched. normally take place. - Replication can be difficult. - Uncontrolled confounding and extraneous variables are presented. Controlled- Watching and - Researcher is able to focus - More likely to be observing recording behaviour in a on a particular aspect of unnatural behaviour as structured environment e.g. behaviour. takes place in an unnatural lab setting. - There is more control over environment. extraneous and confounding - Low mundane realism so variables low ecological validity. - Easy replication. - Demand characteristics presented. Overt- participants are - Ethically acceptable as - More likely to be recording watched and their behaviour is informed consent is given. unnatural behaviour as https://bit.ly/pmt-cc https://bit.ly/pmt-edu https://bit.ly/pmt-cc recorded with them knowing participants know they are they are being watched. being watched. -Demand characteristics likely which reduces validity of findings. Covert- the participants are -Natural behaviour recorded - Ethical issues presented unaware that their hence high internal validity of as no informed consent given. behaviour is being watched results. Also could be invading the and recorded. -removes problem of privacy of the participants. participant reactivity whereby participants try to make sense of the situation they are in, which makes them more likely to guess the aim of the study. Participant- The researcher - Can be more insightful -There's always the possibility who is observing is part of which increases the validity of that behaviour may change if the group that is being the findings. the participants were to find observed. out they are being watched. - Researcher may lose objectivity as may start to identify too strongly with the participants. Non-participant- The - Researcher can be more - Open to observer bias for researcher observes from a objective as less likely to example of stereotypes the distance so is not part of identify with participants since observer is aware of. the group being observed. watching from outside of the - Researchers may lose group. some valuable insight. Observational Designs One problem with carrying out observations is that observer bias is easily presented. This is when an observer’s reports are biased by what they expect to see. A solution to this problem is checking the inter observer reliability of the observation. This is done by many researchers conducting the observational study , their reports are then compared and a score calculated using the formula :- Total number of agreements / total number of observations x 100. The score that shows high inter observer reliability is any score above 80%. There are different types of observational designs and each has their strengths and weaknesses :- Design and description Strengths Limitations Unstructured- consists of - More richness and depth of - Produces qualitative data continuous recording where detail. which is more difficult to https://bit.ly/pmt-cc https://bit.ly/pmt-edu https://bit.ly/pmt-cc the researcher writes record & analyse. everything they see during the - Greater risk of observer observation bias e.g. only record ‘catch the eye’ behaviours. Structured- Here the - Easier as is more - Not much depth of detail. researcher quantifies what systematic. - Difficult to achieve high inter they are observing using - Quantitative data is collected observer reliability as filling predetermined list of which is easy to analyse and the predetermined lists in is behaviours and sampling compare with other data. subjective. methods. - There is less risk of observer bias. Whilst conducting structured observations, behavioural categories can be used. This is when a target behaviour which is being observed is broken up into more precise components which are observable and measurable e.g. aggressive behaviour can be broken down to - shouting, punching, swearing etc. When forming a behavioural categories list, it is important to make sure that behaviours do not overlap with other behaviours, so very similar behaviours should not be listed e.g. grin and smile. They should be clearly operationalised. During structured interviews there are different types of sampling methods: Method and description Strengths Limitations Time sampling- this is the - It reduces the number of - The small amount of data recording of behaviour within observations that has to that you collect within that a timeframe that is made so it is less time time frame ends up being pre-established before the consuming. unrepresentative of the observational study. observation as a whole. Event sampling- this involves - It is good for infrequent - If complex behaviour is being the counting of the number behaviours that are likely to observed, important details of times a particular be missed if time sampling of the behaviour may be behaviour is carried out by was used. overlooked by the observer. the target group or individual - If the behaviour is very you are watching. frequent, there could be counting errors. - It is difficult to judge the beginning and ending of a behaviour. Correlations A correlation is a mathematical technique that is used to investigate an association between two variables which are called co-variables. Correlations differ to experiments as :- The variables are simply measured, not manipulated like in experiments. https://bit.ly/pmt-cc https://bit.ly/pmt-edu https://bit.ly/pmt-cc Only an association is found, no cause-and-effect relationship found hence the terms DV and IV are not used. During correlational studies correlation coefficients are calculated. This value determines the strength and the relationship between two variables. This doesn’t necessarily mean that one variable is causing another, but that there is a relationship of some sort. There are various relationships which can be shown between the co-variables :- Negative correlation - when one variable increases the other decreases. When the data is presented on a scattergram the line of best fit has a negative gradient. It has a correlation coefficient of less than 0. Positive correlation - when one variable increases the other also increases. When the data is presented on a scattergram the line of best fit has a positive gradient.It has a correlation coefficient of more than 0. Zero correlation - no relationship is found between the co-variables. When the data i s presented on a scattergram, no line of best fit can be drawn as the points on the scattergram are random. It has a correlation coefficient equal to 0. Image Source Curvilinear relationship- as one variable increases, so does the other but only up to a certain point after which as one variable continues to increase the other begins to decrease. Ona graph this forms an inverted U shape. An example of such a relationship is shown by the Yerkes-Dodson Law from the topic of Memory which shows how anxiety affects eyewitness testimony. Just as you have hypotheses for experiments researchers also state hypothesis for correlational studies. A directional hypothesis states whether there will be a negative or positive correlation between the co-variables being studies whilst a non-directional hypothesis only states there will be a correlation but the type is unknown. https://bit.ly/pmt-cc https://bit.ly/pmt-edu https://bit.ly/pmt-cc Strengths and Limitations of Correlations: Strengths Limitations - They can be used as starting points to - It is difficult to establish a cause and effect assess patterns between co-variables before relationship, really only an association is committing to conducting an experimental found. study. -The third variable problem is presented - - Quick and economical to carry out. this is when there is a chance that there is - Secondary data can be used in the another variable, a third variable which the correlational study which makes it even less researcher is unaware of that is time consuming. responsible for the relationship between the co-variables. - Lastly, correlations tend to be misused or misinterpreted especially when made public by the media - correlation is often presented as causation. Data Analysis: Types of Data Type and description Strengths Limitations Qualitative data- data which - More richness and depth of - Difficult to analyse. is displayed in words , is detail. - Difficult to make non-numerical. - Allows participants to further comparisons with other data. develop their opinions hence - Researcher bias presented has greater external validity. as conclusions rely on the - A more meaningful insight subjective interpretations of into the participants’ views is the researcher (interpretative achieved. bias). Quantitative data- data that - Can be analysed statistically - Lack of depth in detail. is displayed numerically, not so converted to graphs or - No meaningful insight into in words. charts. participants' views. - This makes it easy to make - As participants are not able comparisons with other data. to develop their opinions the results have low external validity. Primary data - this is when -Targets the exact information - Requires time and effort. information is obtained first which the researcher needs, - Can be expensive. hand by the researcher for so the data fits their aims and an investigation. objectives. Secondary data - this is when - Expensive - It may be likely that the data information is collected by - Data is accessed so requires is outdated or incomplete. someone else other than the minimal effort to collect. - The data may not be researcher yet is used by the reliable- the researcher was https://bit.ly/pmt-cc https://bit.ly/pmt-edu https://bit.ly/pmt-cc researcher for their not there when the study was investigation. Also known as conducted so is likely to be ‘desk research’. unsure of the validity of the results. Meta-analysis - this is when a - More generalisability is - Publication bias such as the researcher combines results possible as a larger amount of file drawer problem may be from many different studies data is studied. presented- this is when the and uses all the data to form - The researcher is able to researcher intentionally does an overall view of the subject view the evidence with more not publish all the data from they are investigating. confidence as there is a lot of the relevant studies but it. instead chooses to leave out the negative results. This gives a false representation of what the researcher was investigating. Data Analysis: Descriptive Statistics Descriptive statistics are the use of tables, graphs, and summary statistics to analyse data. Measures of central tendency These measures refer to any measure which calculates an average value within a set of data. Measure Calculation method Strengths Limitations Mean- arithmetic Total of all values in a - Makes use of all - It is influenced by average. set of data is divided values. outliers (extreme by the number of - Good for interval scores) so it can be values. data. unrepresentative. Median Arrange data from - Not affected by - Not as sensitive as lowest to highest then extreme scores. mean, does not use find the central value. - Good for ordinal all data. data. Mode The most frequently - Useful for nominal - Is not useful when occurring value in a data (data in there are several set of data. categories). modes. https://bit.ly/pmt-cc https://bit.ly/pmt-edu https://bit.ly/pmt-cc Measures of dispersion These measures refer to any measure that calculates the variation in a set of data. Measure Calculation method Strengths Limitations Range Minus the lowest - Easy to calculate. - Affected by extreme score from the highest values. score. - Does not use all data. Standard Deviation The square root of the - Precise measure - Difficult to calculate. (SD) variance calculates where all data values - Affected by extreme SD. A low SD means are taken into values. that more data is account. clustered close to the mean hence there is less data spread Presentation and Display of Quantitative Data There are various ways of representing data: Summarising data in a table One of these ways is summarising data in a table. This is usually not in the form of raw scores but the data has been converted into descriptive statistics for example of the form below :- Table showing the mean and mode of scores of a memory test Condition A Condition B Mean 35 67 Mode 30 34 Below the table there is usually a description of what the table’s data means. Bar Charts This way of representing data allows for differences in data to be seen more clearly. They are used for discrete data, which describes data that has been divided into categories. The bars do not touch each other which shows that we are dealing with separate conditions.The amount of frequency for each category is plotted on the y-axis (vertical axis) whilst the categories (below these are condition A and B) are plotted on the x-axis (horizontal axis). https://bit.ly/pmt-cc https://bit.ly/pmt-edu https://bit.ly/pmt-cc Histograms In this form, the bars touch each other unlike in bar charts and this represents that we are dealing with continuous data rather than discrete. Therefore the x-axis has equal sized intervals of one category (e.g. scores of an english test in intervals 0-10, 11-21, 22-32, etc.) whilst the y-axis represents the frequency (the number of people that score each mark). Line graphs This form also represents continuous data , whereby points are connected by lines to show the change of values. As per usual, the IV is plotted on the x-axis while the DV is plotted on the y-axis https://bit.ly/pmt-cc https://bit.ly/pmt-edu https://bit.ly/pmt-cc Scattergrams These are used to show associations between co-variables rather than differences hence we came across them in the correlations topic. Either of the co-variables can occupy the x-axis or the y-axis, and each point displayed on the graph coincides with the x and y position of the co-variables. Distributions Normal distribution is a symmetrical pattern of frequency data that forms a bell-shaped pattern. A skewed distribution is a spread of frequency data that is not symmetrical, instead the data all clusters to one end. There are two types of these :- Positive skew whereby most of the distribution of data is concentrated on the right. Negative skew whereby most of the data distribution is concentrated on the left. https://bit.ly/pmt-cc https://bit.ly/pmt-edu https://bit.ly/pmt-cc Peer Review AO1 Peer review is the assessment of scientific work by experts in the same field, it is done to make sure that all research intended to eventually be published is of high quality. Main purposes of peer review: To know which research is worthwhile hence funding can be allocated to it. To validate the relevance and quality of research. This is important to prevent fraudulent research from being released to the public. To suggest possible improvements or amendments to the research study. AO3 Anonymity is a problem; reviewers sometimes use it to settle old scores or bury rivals, especially if they're competing for funds. This means that anonymity affects the objectivity of reviewers. Due to this, some journals have started doing open reviewing to avoid this problem. There is publication bias involved in peer review. Editors tend to prefer to publish ‘headline grabbing’ findings and positive results. This brings about the file drawer problem whereby negative results are intentionally not published. All this causes there to be a misconception of the current state of psychology. It can be difficult to find an expert. Smith (1999) argues that because of this a lot of poor research is passed as the reviewer didn't really understand the work. In peer review, any research that opposes mainstream theories tends to be suppressed. This means that established scientists’ work is more likely to be published and the new and challenging ideas are usually rejected. This means that the rate of change in scientific fields is slowed down. Fraudulent research can be long-lasting which is a large problem. An example of this is when the MMR vaccine link to autism was found by Andrew Wakefield (1998). This finding had implications as caused the number of measles cases to increase. It was later found that the research was fraudulent but even now there are still many people aware of these said risks and still anti-vaccine for MMR. https://bit.ly/pmt-cc https://bit.ly/pmt-edu https://bit.ly/pmt-cc The Implications of Psychological Research for the Economy The implications that research has refers to how what we learn from psychological research influences our country’s economic prosperity. The economy is the state of the region's activities of producing or consuming goods & services. Absence from work costs the economy an estimate of 15 billion pounds a year and this absence is mainly due to mental illness e.g. stress, anxiety. For such problems, psychology research has been able to present solutions to them and this expresses why psychology research is important for the economy. From the various AS and A2 topics we have learnt , research in these topics has had implications for the economy: Topic Links to specific areas Economy Psychopathology Treatments - Cognitive - Workers able to return to Behavioural Therapy and work. Rational Emotive Behavioural Therapy for depression, drug therapy for OCD. Attachment Role of the father - Tiffany - Mothers can return to work. Field (1978) found that fathers - More flexible working can take on the role of being a arrangements within families. primary caregiver. - Can maximise their income and effectively contribute to the economy. Social Influence Social influence leading to - Health campaigns. social change - Minority - Unions strike- make working influence, appealing to NSI, conditions better. disobedient models. - Environmental campaigns- like getting companies to reduce their waste and use of non-renewable energy. Memory Eyewitness testimony - How - Led to police using the leading questions or post cognitive interview which event discussion can affect reduces wrongful convictions eyewitness testimony. hence reduces waste of money and space in jail. Case Studies A case study is a detailed study into the life of a person which covers great detail into their background. It looks at the past and present behaviour of an individual to build up a case history hence provides qualitative data. Examples from psychology topics we have learnt :- The case study of HM from the memory topic. HM knew how to tie shoelaces but couldn't remember stroking a dog. This showed us his procedural memory was intact but https://bit.ly/pmt-cc https://bit.ly/pmt-edu https://bit.ly/pmt-cc his episodic memory wasn’t. This showed us that there are different types of long term memory and these are stored in different parts of the brain. Little Hans case study from the approaches topic, in particular the psychodynamic approach. Freud used his case study as evidence for the Oedipus complex. Strengths Limitations - Detailed so able to gain in depth insight. - Not generalisable to wider populations as - Forms basis for future research. data is only gathered from one person. - From studying unusual cases you are able to - Various interviewer biases are presented like infer things about normal usual behaviour of social desirability bias (from the unique humans. person’s side) and interpretative bias ( from - Permits investigation of situations that would the researcher’s side). be otherwise unethical or impractical. - Retrospective studies may rely on memory which can be inaccurate. - They are time consuming and difficult to replicate. Content Analysis Conducting a content analysis involves studying human behaviour indirectly by studying things that we produce e.g. TV adverts, newspapers. This allows us to have insight into the structured values, beliefs and prejudices of our society. How to conduct a content analysis : Identify hypothesis that you will investigate. Create a coding system depending on what you are investigating e.g. 1= male, 2= female. Gather resources. Conduct content analysis and record data in a table. Analyse data which is descriptive and qualitative e.g. using ‘thematic analysis’- allows themes, patterns and trends to emerge in data. Write up a report in the format of a scientific report. Strengths Limitations - Strong external validity as the data is already - Observer bias is presented but it can be in the real world so it has high mundane eliminated by achieving inter-observer realism. reliability. - Produces large data set of both quantitative - Content of choice to analyse can be biased and qualitative data that is easy to analyse. by researcher. - Easy replication. - Interpretative bias - the researcher may - Ethical issues like ‘right of privacy, ignore some things but pay extra attention to confidentiality, informed consent’ are avoided others. as data is already in the public domain. https://bit.ly/pmt-cc https://bit.ly/pmt-edu https://bit.ly/pmt-cc Levels of Measurement Quantitative data can be divided into different levels of measurement, either - nominal, ordinal or interval. Nominal data refers to a type of data that is in the form of categories. It is discrete- one item can only appear in one category. It does not enable sensitive analysis as it does not yield a numerical result for each participant. Ordinal data refers to data which is represented in a ranking form e.g. 1= hates maths, 10= loves maths. There are no equal intervals between each unit. A weakness of it is that it lacks precision as is based on the subjective opinion of people. Interval data refers to the type of data that is based on numerical scales which include equal units of precisely defined size. This is the most sophisticated form of data as it is based on objective measures. It is needed for the use of a parametric test. Appropriate measures for each level of data: Level of data Measures of central Measures of dispersion tendency Nominal Mode n/a Ordinal Median Range Interval Mean Standard Deviation Reporting Psychological Investigations: Scientific Report Psychologists use a particular format to write up their research for publication which is known as a scientific report. A scientific report consists of various sections: Abstract - this part includes a summary of all the key details of the research report. These key details include the aim, hypothesis, method, results and conclusion. It is usually about 150-200 words long and is the part that is supposed to be read to know whether the research study is worth examining any further. Introduction - This includes information of past research on a similar topic whereby relevant theories, studies and concepts are mentioned. At the beginning it tends to be broad but as it continues towards the end the information becomes more specific until the aims and hypotheses of the study are presented. Method - This part includes a description of what the researchers exactly did when they undertook the study. This includes the design, sample collected (specific details e.g. target population, sampling method, demographic data of participants), materials used , procedure (specific e.g. standardised instructions for each participant), ethics etc. There should be sufficient detail included so that any other person is able to read this part of the report and replicate the investigation precisely. https://bit.ly/pmt-cc https://bit.ly/pmt-edu https://bit.ly/pmt-cc Results - This includes all the findings from the study, presented even with inferential and descriptive statistics. If qualitative data is collected then this section may include a thematic analysis. Discussion - This is where the researcher considers what the findings exactly mean for us and for psychological theories. Usually the findings are summarised here then they are discussed in context to the introduction. Limitations thus ways of improving the study and the wider implications it may have for society may also be discussed here. Referencing - This is the last part of the scientific report which is basically a list of all the sources that were quoted or referred to in the report. These can vary - journal articles, websites, books- and full details are given so that a reader is able to find the exact source the researcher was referring to. Books are referenced differently from journals: Books are referenced in this order: author(s), date, title of book (in italics or underlined), place of publication, publisher. E.g. Flanagan, C. and Berry, D.(2016). A level Psychology Year 2. Chettenham: Illuminate Publishing. Journals are referenced in this order: author(s), date, title of article, journal name, journal volume, issue number (all appear in italics or underlined apart from author, date and title of journal article), page range. E.g. Gupta,S.(1991). Effects of time of day and personality on intelligence test score. Personality and individual differences, 12 (11). 1227-1231. Introduction to Statistical Testing: Use of the Sign Test Statistical testing provides a way of determining whether hypotheses should be rejected or accepted. It can tell us whether differences or relationships between variables that have been found during experiments are statistically significant or if they have only occurred due to chance. An example of a statistical test is the sign test. A sign test can only be used for a study that :- Looked for a difference not an association. Used a related experimental design- repeated measures design. Collected nominal data. How to conduct a sign test: Step 1 - State the hypotheses- this includes both the alternative and the null hypothesis. Step 2 - Record data and work out the sign. For example, the sign will be negative (-) if the value has decreased in the second condition but positive (+) if it has increased. If the value has stayed the same , this value will be ignored and the N adjusted to exclude it. Step 3- Find the calculated value for the sign test, S, which is the number of times the less frequent sign occurs. https://bit.ly/pmt-cc https://bit.ly/pmt-edu https://bit.ly/pmt-cc Step 4 - Find the critical value of S - use the calculated N value (which is the total number of values with the ignored values excluded) and p≤ 0.05 which means there's a less than 5% probability that the results occurred by chance. If S ≤ critical value- reject the null hypothesis, there is a significant difference. If S ≥ critical value - accept the null hypothesis, there is no significant difference. Step 5 - State conclusion whereby you refer back to the hypothesis mentioning the IV and Dv and support your conclusion with the exact values of -the critical value, S, N and what p value you used. Choosing an Inferential Statistical Test When choosing an inferential statistical test you have to think about three factors: The design of the study - Did it involve an unrelated design which is of the independent groups design? Did it involve a related design which could be either using the repeated measures or matched pair experimental design? The level of data collected during the study - either ordinal, nominal or interval. Whether a difference or correlation is being measured These factors have been summarised into the table below which shows which statistical tests to use in different situations: Type of data Test of difference Test of association/ correlation Unrelated Related Nominal Chi-square Sign test Chi-square Ordinal Mann-whitney Wilcoxon Spearman’s Rho Interval Unrelated t-test Related t-test Pearson’s R Remembering the table above will help you answer exam questions which can either ask you why a student may decide to use a particular test after providing you with a stem of a research study or may ask you which statistical test is appropriate to use for that study. Use of statistical tests Statistical tests are used to determine whether a significant difference or correlation exists. This is discovered using the calculated value (the result obtained from the statistical test) and the critical value (the numerical boundary that stands between accepting or rejecting the null hypothesis when a hypothesis is being tested). The critical value is worked out from a table of probability values and depends on various factors: whether it was a one or two tailed test, the P value and either the N value or the degrees of freedom value. Rule of R - If there is an R in the name of the statistical test the calculated value has to be gReater or equal to the critical value for the result to be significant. If this is the case then the https://bit.ly/pmt-cc https://bit.ly/pmt-edu https://bit.ly/pmt-cc null hypothesis can be rejected and the alternative hypothesis is supported. If there is no R in the test’s name then the calculated value has to be less than or equal to the critical value for it to be significant. Probability and Significance Significance is a statistical term which lets us know how sure we are about a correlation or difference existing. If significant, we reject the null hypothesis and accept the alternative hypothesis. The difference between these two types of hypotheses is the null (H0) one states ‘there is no difference or correlation between the conditions’ whilst the alternative (H1) one states ‘there is a difference between the conditions’. Probability is a calculation of how likely it is for an event to happen - 0= statistical impossibility and 1= statistical certainty. The usual level of significant in psychology is 0.05. Therefore the p value is usually equal to or less than 0.05 (5%) which means that the probability of the difference in the study’s findings being due to chance is 5% or less so researchers have a 95% confidence level in their results. If there is any risk attached to the research like a ‘human cost’ e.g. with clinical drug trials then the p values is set at 0.01 (1%) instead. Type I and Type II errors When researchers conduct inferential statistical tests they can make either of two types of errors when forming a conclusion from the test:- Type I (optimistic) error is the incorrect rejection of a null hypothesis which is actually true. Researchers claim to have found a significant difference when there actually isn't any (a false positive). Type II (pessimistic) error is the failure to reject the null hypothesis that is false. Researchers claim that there is no significant difference when there actually is one (a false negative). Features of a Science Paradigms & paradigm shifts A paradigm is a set of shared ideas and assumptions within a scientific discipline. A paradigm shift is a significant change in these central assumptions within a scientific discipline, resulting from a scientific revolution. Kuhn (1962) suggests that paradigms are what separate scientific disciplines from non-scientific disciplines. Kuhn also believes that paradigm shifts show progress within a science. In respect to this feature, psychology has too much disagreement and conflicting approaches (e.g.is behaviour biological from genes or from conditioning experiences?) so isn't able to qualify as a science. It is referred to as a pre-science. https://bit.ly/pmt-cc https://bit.ly/pmt-edu https://bit.ly/pmt-cc Theory construction & hypothesis testing A theory refers to a set of general principles and laws which can be used to explain specific events or behaviours. Theory construction takes place through gathering evidence from direct observation during investigations. You should be able to make different hypotheses from a theory, then when the hypothesis is supported the theory is strengthened. If it is not then the theory may need to be revised. Deduction refers to the process of deriving new hypotheses from an already existing theory e.g. Baddeley and Hitch modified the Working Memory Model in 2000 as they added the episodic buffer to the model. Falsifiability Falsifiability is the principle that states a theory cannot be considered scientific unless it allows itself to be proven untrue. Popper (1934) argues that this is a key criterion for a scientific theory. Popper proposed the theory of falsification which states that successful theories that have been constantly tested and supported simply haven't been proven false yet. Sciences that can't be proven wrong are known as ‘pseudosciences’- a good example is Freud’s concepts from the psychodynamic approach like the Oedipus’s Complex. Theories which survive more falsify attempts are seen as the strongest. This theory of falsification explains why when stating hypotheses for an investigation the alternative hypothesis is always accompanied by the null hypothesis. This also explains why we never use the word ‘proves’ in investigations even if the results support the researcher’s hypothesis. The hypothesis-deductive method refers to the process of formulating hypotheses that can either be proved or disproved by experimentation. Replicability Replicability refers to the extent to which scientific methods and their results can be repeated by other researchers across other contexts and circumstances. It is used to assess validity and reliability of results from a research study. Objectivity & the empirical method Objectivity is when all possible biases from the researcher are minimised so that they don't influence or distort the research process. The empirical method is when evidence is collected through making direct observations and through direct experiences. A theory is not able to be scientific unless it can be empirically tested and verified using either the empirical experimental or observational method. Psychology as a science: Supporting arguments Against - Produces intuitive results which are against - Experiment interpretations can be subjective. https://bit.ly/pmt-cc https://bit.ly/pmt-edu https://bit.ly/pmt-cc common sense. - Not all research is generalisable e.g. from - Scientific methods are used in many research case studies. studies giving them scientific credibility. - Psychologists do often make inferences of - Findings from studies do positively impact behaviour rather than directly measuring it , for society & individuals e.g. Cognitive behavioural example this is usual for cognitive Therapy to treat depression. psychologists that infer about cognitive processes from brain scans (Memory topic link here). Ethical Issues & Ways of Dealing with Them Issue Explanation Solution Informed consent Participants must be told the There are various methods of purpose of the investigation dealing with informed consent: (their aims) and about any - Prior general consent- potential risks they may be participants give permission to subject to when taking part in take part in many studies it. This allows them to make whereby one of them involves an informed decision on deception so effectively they whether they want to are consenting to getting participate in the research deceived, study. - Presumptive consent- when a researcher gathers Researchers don’t always opinions from a group like the wish to disclose this participants in the study but information as it could lead to does not inform the actual demand characteristics being participants. Allows demand presented hence result bias. characteristics to be eliminated. - Retrospective- this is when the participants are asked for consent after they have participated in the study. Deception This is the act of deliberately - Debriefing- all participants withholding information from would be debriefed after the participants or misleading study, it can be a written or them during the research verbal debrief. During the study. This is only seen as debrief the true nature of the acceptable when the study must be said and the participants knowing the true participants should be told nature could guess the aims what their data will be used of the investigation or when for. After the debrief the deception will not cause participants have the right to distress. choose to withhold or withdraw their data. Protection of harm Participants must be protected - If the participants have been https://bit.ly/pmt-cc https://bit.ly/pmt-edu https://bit.ly/pmt-cc from physical and subject to any stress or psychological harm. It is the psychological harm, the job of the researcher to make researcher should provide sure of this. counselling if it is required. - A Cost-Benefit Analysis All through the investigation, should be done before a study participants are also reminded is carried out. This is done by that they do have the right to the ethics committee whereby withdraw, especially if the the pros and cons of the study study is causing them harm. are weighed up to determine whether the study will be ethical. This can be difficult and an example of where this was done but went wrong is for Zimabardo’s Stanford Prison Experiment in 1973 (Social Influence topic). Privacy and confidentiality Right of privacy refers to the - Anonymity can be right that participants have to maintained. This is achieved controlling information about by the researchers not themselves- how much is recording any personal details released and how it is used. It of their participants so that can be difficult to avoid none of the results data can invading a participant's be traced back to them. privacy for example if it is a Instead the researchers can field study- these are done in refer to the participants using natural environments. The numbers or initials when right of privacy can extend to writing up the investigation the location of the study e.g. HM case study. whereby the institution is not -The participant should be named. reminded during both the Confidentiality refers to the briefing and debriefing of the right participants have which investigation that their data concerns any personal data of will be protected theirs being protected. https://bit.ly/pmt-cc https://bit.ly/pmt-edu https://bit.ly/pmt-cc Self-Report Techniques & Design Self-report techniques refer to any sort of method where a person is asked to give their opinions, feelings, experiences and behaviours in relation to a particular topic. There are two types of these non-experimental investigations: Questionnaires - These assess a person’s thoughts or experiences through a number of different written questions. Interviews - This involves a live encounter where a set of questions is asked by an interviewer to an interviewee to assess their thoughts or experiences. Questionnaires There are two types of questionnaires: Type and description Strengths Limitations Open Question - This is - Rich in depth and detail. - Difficult to convert to when the questions are - Useful for sensitive topics as statistical data hence more phrased in a way that the participants can elaborate on difficult to analyse. participant is free to answer their answers. however they like, there are no restrictions. This type collects qualitative data. Closed Question - In -Easy to analyse data and - Lack of depth and detail. contrast, this type of compare with data from - Can be limiting which can be questionnaire consists of elsewhere. frustrating for participants. questions which restrict you to a fixed number of responses. This type collects quantitative data. Examples: - Likert scale- the respondent indicates agreement with a statement, ranges from agree to strongly agree. - Rating scales- a rating scale works in a similar way but gets respondents to identify a value that represents their strength of feeling about a particular topic. - Fixed choice scales- the question includes a list of possible options and respondents are required to indicate those that apply to them. https://bit.ly/pmt-cc https://bit.ly/pmt-edu https://bit.ly/pmt-cc There are various strengths and limitations of questionnaires: Strengths Limitations - Cost-effective. - Difficult to know whether the target population - Gathers large amounts of data quickly. it was intended for answered it e.g. if it is - The researcher does not need to be present. online. - They are easy to analyse. - They take a long time to design. - As responses can be anonymous this usually - It is difficult to assess the validity as biases means participants are more open. such as social desirability bias (when the participant wants to present themselves in a positive light so is not truthful) are presented. - Participant bias presented from factors such as time, age, gender. - Response bias presented e.g. acquiescence bias whereby participants simply agree with all the questions, instead of putting effort into considering an answer for each question. Construction of questionnaires: There are various factors that need to be thought about when designing questionnaires :- Clarity - the questions should be phrased in such a way that it is clear for the respondent on what answer is needed from them. Avoid overuse of jargon, emotive language, double-barrelled questions, double negatives and leading questions. All these can cause biases which affects the validity of the results. Sequencing questions - easy ones can be first then followed by the harder ones. This allows a build up of confidence in each participant. Filler questions - these are questions which have nothing to do with the aims of the investigation and are put in to distract the participant from guessing the real aim of the study. Therefore these eliminate demand characteristics. Pilot study - can be carried out to ensure that the questionnaire is suitable and if not amendments and improvements can be made. Interviews There are two main types of interviews: Type and description Strengths Limitations Structured - Involves a set of - Standardisation is - Interviewer bias which can be predetermined questions possible. presented through aspects such being asked during the - Easily replicable. as body language, listening skills, interview. The interviewer - Can make comparisons when to ask a question and asks the questions and for between participants interpretative bias (how answers each waits for a suitable easily which is a strong are recorded). https://bit.ly/pmt-cc https://bit.ly/pmt-edu https://bit.ly/pmt-cc response. benefit for job interviews. - Social desirability bias. - Not being able to elaborate can be frustrating for participants. Unstructured - There are no - Lots of data is collected - Skilled interviewers needed. predetermined questions , with more depth and - Interviewer bias also presented. instead questions develop detail. - Social desirability bias. as the interview goes on. - As can be tailored to - Difficult to make comparisons This allows for questions be individuals they can between participants. tailored to individuals and is provide more insight. - The analysis of data is difficult more free flowing. as may have to sift through a lot of irrelevant data. There can also be semi-structured interviews whereby most of the questions are already set up but the interviewer is free to ask any follow up questions on certain answers. Construction and design of interviews: Recording information - this can be done in various ways e.g. writing down answers, using a video recorder, using an audio recorder. Ethical issues - Informed consent is needed from the participant for the researcher to obtain and keep the data. The participant should be reminded that their answers will be kept confidential. Location - A quiet room away from other people is the most appropriate as this location is likely to get the participant to feel comfortable and open up. Neutral questions - These are usually started with to make the participant feel relaxed and help establish a rapport. Reliability Across all Methods of Investigation Reliability is a measure of how consistent the findings from an investigation are. Why is it important? To ensure the DV is being measured accurately. To ensure that over periods of time, the outcome is still the same. To ensure that all the conclusions made are accurate and valid as if not can have implications for theory development. There are various types and ways of assessing each: Type and description Ways of assessing Internal reliability- describes how consistent - Split half method - Randomly select half of the something is within itself. questions and put them in one form then do the same for others. Theses two forms of the same test are then done separately and should yield the same score,have a correlation coefficient of ≥ 0.80 https://bit.ly/pmt-cc https://bit.ly/pmt-edu https://bit.ly/pmt-cc External reliability- this is when consistent -Test-retest method - the researcher administers results are produced regardless of when the the same test on the same person on different investigation is used or who administers it. occasions. The results should yield a correlation coefficient of ≥ 0.80. Sufficient time should be left between the test and retest so participants cannot recall their answers, and this time in between should not be too long as the person’s attitudes may change. - Inter-observer reliability - refers to the extent to which there is agreement between two or more observers involved in observing behaviour. This eliminates subjectivity bias and may either be carried out in a pilot study or reported at the end of the study. It is calculated by the formula: Total number of agreements ≥ 0.80 = High Total number of observations inter observer reliability Ways of improving reliability in: Questionnaires A questionnaire that produces low test-retest reliability may require some of the items to be ‘deselected’ or rewritten. One solution might be to replace some of the open questions where there may be more room for misinterpretation, with closed, fixed choice alternatives which may be less ambiguous. Interviews For interviews, probably the best way of ensuring reliability is to use the same interviewer each time. If this is not practical or possible, all interviewers must be properly trained, so for example, the interviewer should not ask leading or ambiguous questions. They should all be able to structure their interviews in a ‘certain manner’ which can be followed by all to ensure that everything is similar. This is more easily avoided in a structured interview where the interviewer’s behaviour is more controlled by the fixed questions. Interviews that are unstructured and more ‘free-flowing’ are less likely to be reliable. Experiments Lab experiments are often described as being ‘reliable’ because the researcher can exert strict control over many aspects of the procedure, such as the instructions that participants receive and the conditions within which they are tested. This control allows for the experiment to be designed to be replicable, so that if someone else were to repeat it, they would get similar findings (ideally). https://bit.ly/pmt-cc https://bit.ly/pmt-edu https://bit.ly/pmt-cc Such control is often more achievable in a lab than in the field, since in the field variables are much more difficult to control and the researcher doesn’t have access to it throughout the procedure, so they just have to ‘make-do’ with the situation in hand. This is more about precise replication of a particular method rather than demonstrating the reliability of a finding. One thing that might affect the reliability of a finding is, if participants were tested under slightly different conditions each time they were tested. Observations The reliability of observations can be improved by making sure that behavioural categories have been properly operationalised, and that they are measurable and self-evident. For instance, the category ‘pushing’ is much less open to interpretation than ‘aggression’. Categories should not overlap for example, ‘hugging’ and ‘cuddling’ and all possible behaviours should be covered on the checklist. If the categories are not operationalised well, are overlapping, or absent different observers have to make their own judgements of what to record where, and may well end up with differing inconsistent records. Validity Validity refers to the extent to which results of a research study are legitimate. There are various types of validity and ways of assessing them: Types and description Ways of assessing Internal validity - this is whether the outcomes - Face Validity - this is when a measure is observed in an experiment are due to the scrutinised to determine whether it manipulation of the IV and not any other factor. appears to measure what it is supposed It is influenced by :- to. This can be done either through simply Confounding and extraneous variables. looking at it or passing it to an expert to Participant variables and demand check. characteristics. Investigator bias. - Concurrent validity- this refers to the extent to which a psychological measure External validity - This relates to factors outside compares to a similar existing measure. the investigation - is it generalisable to other The results obtained should either match or settings, populations & eras. There are different be closely similar to the results of the well forms of external validity: established and recognised test. Ecological validity- This is the extent to - Predictive validity- this refers to how well which findings can be generalised to other a test can predict future events or situations and settings. behaviours Temporal validity - Generalisability to other E.g. how childhood attachment measured historical times and eras using the strange situation are able to predict Population validity - Generalisability to how the child will grow up to behave in different populations of various ages, adulthood (from Attachment topic). https://bit.ly/pmt-cc https://bit.ly/pmt-edu https://bit.ly/pmt-cc genders and cultures Ways of improving validity in: Experimental research Using a control group in experimental research means that the researcher is better able to assess whether changes in the dependent variable were due to the effect of the independent variable. For example, in a study looking at the effectiveness of a therapy, a control group who did not receive therapy means that the researcher can have greater confidence that improvements were due to the effects of the therapy rather than, say, the passage of time. - Experimenters may also standardise procedures to minimise the impact of participant reactivity and investigator effects on the validity of the outcome - The use of single-blind and double-blind procedures are designed to achieve the same aim - In a single-blind procedure participants are not made aware of the aims of the study until they have taken part (to reduce the effect of demand characteristics on their behaviour) - In a double-blind study, a third party conducts the investigation without knowing its main purpose either (which reduces both demand characteristics and investigator effects and thus improves validity) Questionnaires Many questionnaires and psychological tests incorporate a lie scale within the questions in order to assess the consistency of a respondent’s response and to control for the effects of social desirability bias. Validity may be further enhanced by assuring respondents, that all data submitted will remain anonymous. Observations Observational research may produce findings that have high ecological validity as there may be minimal intervention by the researcher. This is especially the case if the observer remains undetected, as in covert observations, meaning that the behaviour of those observed is likely to be natural and authentic. In addition, behavioural categories that are too broad, overlapping or ambiguous may have a negative impact on the validity of the data collected. Qualitative methods Qualitative methods of research are usually thought of as having higher ecological validity than more quantitative, less interpretative methods of research. This is because the depth and detail associated with case studies and interviews, for instance, is better able to reflect the participant’s reality. https://bit.ly/pmt-cc https://bit.ly/pmt-edu https://bit.ly/pmt-cc However, the researcher may still have to demonstrate the interpretative validity of their conclusions – this is the extent to which the researcher’s interpretation of events matches those of their participants. This can be demonstrated through such things as the coherence of the researcher’s reporting and the inclusion of direct quotes from participants within the report. Validity is further enhanced through triangulation – the use of a number of different sources as evidence for example, data compiled through interviews with friends and family, personal diaries, observations etc. https://bit.ly/pmt-cc https://bit.ly/pmt-edu https://bit.ly/pmt-cc

Use Quizgecko on...
Browser
Browser