QuantNotes PDF

How Do lale know What tale know There are issues of cognitive conservatism memory failures attribution errors Also cognitive biases halo effect truth bias confirmation bias Experience Authority Tradition Intuition But many oftheserely on...

How Do lale know What tale know There are issues of cognitive conservatism memory failures attribution errors Also cognitive biases halo effect truth bias confirmation bias Experience Authority Tradition Intuition But many oftheserely on cognitive logicprocesses a flawed system Epistemology What are the necessary sufficient conditions of knowle Empiricism Empirical knowledge is based on Systematic Observatio Social Science Epistemology What counts as knowledge in the social science That which can be reliably observed Social Scientific Inquiry Methods Different questions require different methods Scientific Quantitative Surveys Experiments Interpretive Qualitative Interviews Focusgroups Ehnography Participant Observation Just means explanation Quantitative DeductiveTheory Goal is to create Testable Falsifiable Hypotheses Theoretical constructs must be observable Interpretive InductiveTheory Goal is to create plausible interpretations of socialdat Hypotheses Testing Hypotheses must propose same relationshipbetweenvariabl Typically two forms communication apprehension is H1 X is related to y related to the size ofaudience  H2 Group A will be different from Group B on how much Canadians groupA and Brazil variable group B trusts their government variable time on social media satisfaction w friendships L Satisfaction Quantitative Methodology A question of methodology A quantitative methodology is focused on producing knowledge that is unbiased Replicable Resulting methods tend to produce quantifiable resul And hopefully generalizable results I  note for assign Confidence In Quantitative Findings What are we studying conceptualization How are we studying it operationalization Who is participating sampling Is our method of datacollection clear replicability Variables Any entity that can take on different value The building blocks of research design Everything about our quantitative research design choices depends upon the variables that we want to measure Conceptualization Refinement specification of abstract concepts Conceptualizations are working agreements Not the same as a dictionary definition May vary from study to study Indicators absence or Things that indicate the presence of our concept Dimensions subgroups of indicators Language The Problem Leadership Satisfaction Love Culture Competence Control Apprehension Affection What are possible indicators for your concept Example Communication Apprehension Nervousness speechlessness Butterflies in stomach sweaty worry filled pauses Does your variable have more than one dimension Cognitive Physical Behavioural Anxiety Trembling hands Speechlessness fear butterflies Disfluency Worry Hives Blotchy skin filled Pauses How can you be confident the findings in The reason for this question is that all studies are wrong There's a whole amount of population but for research they get just a sample Central Limit Theorem mean of the 41ˢᵗ IThe the population Remembthescreentime There's a real life mean but just sampling a few will not give you that Here's the idea mini researches final result but then a collectio done with samples of them can give a can only give a semi correct fully correct result result result 14 result 24 result 34 Jesuit 4 result 50 A normal distribution The peak is the mean There are values clustered around the mean It is important to consider many studies and not be swayed by a particular study no bias VARIABLES If something be different from person to person can then it could be a variable that we can quantify use in data analysis The level of satisfaction in online relationships How It many people click on online ads would depend on what ads people mostly click on How much someone weighs The weight is the entity Back to the example of phone usagethethere are entity various things to consider like how connected the person feels And furthermore what brings that connection is it the amount of friends the amount of supportthey feel even with the terms friends supportyou still need to go deeper Types Levels of Variables Categorical NominalVariables They are mutually exclusive but not ordered Hair Color Gender Job There's no scale to them Ordinal Variables They are ordered categories Data are ranked particulardegree butthere in a isn't a specific difference between data points t.ie of dnt credits to be consider Internal Variables Data is ranked in terms of degree and the differen between the two values of the variable is meaning But there is no true 0  Temperature in Celsius 10 12 130 140 150 11 Thereof 0 in temperature but it doesn't mean the Strongly Disagree Strongly Agree scales Ratio Variables Data is ranked in terms of degree and the differen betweenÉwo values of the variable is meaningful And the scale has a meaningful 0 Temperature in a Kelvin scale Height Ialeight Age Always measure data at the most sophisticated variable level possible Please select the following categories to indicate your age 18 24 25 30 31 39 40 45 this shows that examples This is ORDINAL level data of a simpler level can be presented in a more categorical level Please record your 338 age in months this shows that even simpler This is RATIO Level databut Level data can bemaderema made simpler more simple and still At A A Bt B B still ordinal level in that level You can always categorize ratio level data but you can't turn categorical data into higher level variable CONCEPTUALIZATION OF VARIABLES Abstractions Some variables don't need much conceptualization theyjust are Like age height time spent on phon But other variables very abstract and so we need are to think carefully about the conceptualization of our research Language often abstract and arbitrary is Thus Perceptions of variables can also be abstract and arbitrary but when you get to German Shephard The word dog you get less abstract Indicators Things that indicate the presence or absence of that concept what does this concept mean to the average person what words would I use to better explain it The indicators themselves is not the variable its more like the presence of many indicators shows the variable Here is a finished conceptualization of Communication Apprehension Communication Apprehension is a person's response to being uneasy about an upcoming public presentati People experiencing it may feel anxious or fearful has physical arousal symptoms like a faster heartbeat orhives and engage in filled pauses speechlessness or disfluencies CONNECTING VARIABLES TO QUESTIONS Research Questions They propose that two variables or more are related but we may not be sure about the relationships between those variables They are general than hypotheses more What personality traits might predict being willing the variables Terri after matching on Tinder What behaviors are related to maintenance in first year college students newly formed a casual and b close friendship networks Hypotheses Texted relational maintenance strategies will be correlated positively with relational satisfaction Variable X Texted relational maintenance strategic Variable t Relational satisfaction Type Variable X is related to Y Participants who abstain from social media will spend more time engaging in mediated informational and entertainment activities Type Group A will be differentfrom Group B on variable X Group A Participants whoabstainfrom socialmedia Group B Participants who don't Variable X level of engagement in mediated informational entertainment activities EXPERIMENTAL DESIGN EXPERIMENTS Introduce some action manipulation treatment by the experimenter Observe the consequence of that action Benefits Control Establish Casuality Criteria for casuality X is related to Y temporally precedesY X and Y are not related through some tab their relationship is not spurious TRUE EXPERIMENTAL DESIGN True Experiments involve A Control group also called experimental or An Experimental group treatment condition Random assignment to these groups Why control group a Use of a control group helps researchers note that  the existence of and control some participant relate threats to internal validity Why random assignment Helps assure that experimental control groups are equivalent on any and all characteristicsupon which they could be compared Eliminates the threat of selection bias to the independ Participants do not self select exposure variable The Double Blind Experiment Participants Research Assistants do not have knowledge of whether a participant is in the controlo experimental group Helps eliminate researcher related threats to internal validity Manipulation Checks Checks to make certain that the operationalrate of the 14 was what the researcher intended CONTROL GROUP DESIGN ILLUSTRATION Pretest Posttest R O X 02 R 03 04 Posttest Only R X O R 02 Solomon Four Group R O X 02 R 03 04 R X 05 R 06 QUASI EXPERIMENTAL DESIGN Experiments that lack full experimental control not randomly assigned Typically involve some type of comparison group GROUP DESIGN ILLUSTRATION group design NÉeaIlenÉCÉntrf 03 04 MultiplesIgnseries QQ 03 1 040506 1070809 010011 012 PRE EXPERIMENTAL DESIGNS Experiments that lack random assignment to experimental and control group One shot case study 0 One Group pre test posttest O 02 Static Group Comparison I I 02 SAMPLING TERMS Who is filling out the qSessions falho as participaty in the experiment What Is A Sample A subset of elements or units selected from a population Population what is it Theoretically specified aggregation of study adf.EE mea ridaround The entire group to which you wish to generalize the results found in your sample Youneed to be careful about choosing your sample don't just go for who is accessible Sampling Elements A single case in the population Often people Sample Generally impossible to study everyone so we study a smaller sample Goal is to get a sample that is representative of the population we wish to study Norm probability samplings Convenient sampling PROBABILITY SAMPLING The gold standard for sampling in social scientific research AKA Random Sampling Each population member has an equal chance of being selected for the sample No population member's chances of being in the sample is dependent upon the status of some other population member  EE Establish yn EEEY a of Sample elements Sampling Frame List of units composing a population from which the sample is selected Sample Size Bigger is better but researchers must manage resources A sample is big enough when it can detect a statistical effect at a desired significance level Can use previous research as a guide Systematic Sampling Every Kth element in the total list is chosen There is a sampling interval Distance between elements selected in the samp Population size Sample size 44 6 7 3 I  you have to be careful that the list is not ordered in a certain way Stratified Sampling Population is divided into strata Then samplewithin strata until needed numbers are met BENEFITS OF RANDOM SAMPLING Less likely study outcome is influenced by the researcher Sample more likely to be representative of the population Sample characteristics should approximate population characteristics NON PROBABILITY SAMPLING Commonly called Convenience Sampling 1 1 8 9 Why Do Non Probability Sampling lathile probability sampling theoretically is best it issues with gettingthe is also practically impossible sampling frame There are better convenience samples worse What we categorize as good or bad convenience samples depends on what population you are generalizing to wth The more representative we can argue that the sample is the better Convenience Sample Reliance on available participants Closest people you can find Canintroduce several biases Nonrepresentative sample Self selection Purposive Sample Also called judgemental sample Closest specific people you can find mak Research questiondrives sample choices s p Snowball Sample Collect data from a few known members of a target population and then ask those peopleto provide other members of the population Can be useful if population is difficult to access and or there are gatekeepers Participants are not independents Quota Sampling Participants are selected non randomly on the basis of their known proportion in the population Can get a more representative sample May be non random on confounding characteristics Are non probability samplings ok Pragmatic Approaches To Increase ExternalValidity Specify the Consider Measure population Bias D TIPS FOR WRITING SURVEY MEASURES Measure data at the most sophisticated level possible Write more items than you think you will need Some items may be bad items EVERY item must reflect the variable Items should be short Avoid Double Barreled Questions Ex I try to be cheerful and pleasant TIPS FOR CREATING MEASURES Questions should be clear Avoid jargon slang language only a fewundersta Use language the respondentunderstands Keep age education levels in mind Be specific prease Provide people with specific frames Times Relationship Whenconsidering Avoid using always or never Although these do make reasonable anchor points in some cases know how you plan to analyze the data OPERATIONALIZATIONS OPERATIONALIZATIONS Development of specific research procedures that will result in empirical observations representing this concepts in the real world MEASUREMENT Careful deliberate observations for the purpose of describingobjects events in terms of the attributes composing a variable TURNING CONCEPTS INTO NUMBERS Transforming ideas into measures Using measures to assign numbers tothoseideas How do we make something like love or communicate apprehension into a number INHAT KINDS OF OPERATIONALIZATION DO WE SEE IN THE SOCIAL Sciences Direct Observation Observing in the theatre the pla Train observers coders people touch to showaffection visual detectors to checkwhen Physical measurements people view in a socialmediapa Environmental Residue Textual and Narrative Data or Survey Self Report Data OPEN ENDED QUESTIONS Respondents provide their own answer to a questio Benefits Answer choice not constrained by the research is Data may be richer more detailed Drawbacks Data must be coded Answers might not be relevant CLOSED ENDED QUESTIONS Respondents choose answers from a provided list Benefits Uniform response Already in numeric format Drawbacks Structure of response not match respondent may experiences May not capture the richness of the data DEVELOPING MEASURES What the indicators of your variable are Craft items around these indicators Use multiple items to create a composite measure likert scale semantic differential MEASURE CREATION GUIDELINES Is the respondent willing able to answer the question Social desirability Bias Do they have the knowledge or experience Can they remember this thing REHABILITY VALIDITY These are both importantconcepts in research methods RELIABILITY This isabout consistency If you do the same thing the same way would ask partiupantsthe same questio the answers be similar It is often considered in measurement or operationalizat VALIDITY This is about accuracy Are the findings of the study reflective of what actually goes on in the real world is a little This because if we knew the real state iffy of the world there would be no reason to do research How To DETERMINE RELIABILITY AND VALIDITY We really can't fully know if a study is reliable or valid but there are ways to try MEASUREMENT RELIABILITY Would the same technique applied repeatedly to the same object yield the same result each time An example of the scale If scale gives you a kg b kg c kg for stepping on a it 3 consecutivetimes in a minute you change it because it is not reliable However if you step on the scale and it gives you the same results but the result is not correct then it isn't valid You can have measures that are reliable but not valid but you can't have valid but not reliable measures RELIABILITY TECHNIQUES Test Retest produce the same results Test Retest when done on the same sample Consistency in results suggests reliability of scale 10 indicators Internal Consistency scale of 10 itemsmeasuringthesame Split half reliability of 35.8cm Randomly choose two subsets of items Subset should be highly correlated Item total reliability Scores on individual items should positively correlate with the total scores on the scale Reliability Coefficients Can capture the agreement betweenitems with  a reliability coefficient Most Cronbach's α common is Ranges from 0.00 to 1.00 Researchers hope to produce values close to 1.00 0 70 accepted a general cutoff for reliability The more items the less Cronbach's α Confirmatory Factor Analysis Adequate fit VALIDITY OF SURVEF MEASURES MEASURE The measure accurately reflects the concept it intends to measure CONTENT VALIDITY Does the measure cover the range of meanings of indicators and dimensions included within the concept Includes Face validity Does a given measureits face represent the on conceptualization of the variable Can be tested by asking a sample similar to the study population if th feel the questions  represent the concept Expert Panel validity Using expert opinion to decide if the measure represents the conceptualization of the variable Experts might Evaluate the measure post construction Help create the measure CRITERION VALIDITY Is the measure assessing the core criteria of the concep Includes Predictive Validity Does this measure predict future behavior some call this Convergentvalidit Concurrent validity Do participants scores on the measure matchscores on similar measures Use of the set diagrams with 2 circles of self other to measure closeness CONSTRUCT VALIDITY Doesthe construct fit into the universe of constructs in the way that we theoretically expect it to Includes  Convergent validity Measure is positively correlated with theoretically related variables measures on love like Discriminant validity Measure is negatively correlated with theoretically different variables measures on love hate You wouldn't necessarily see all these types of validity in one study THREATS TO EXPERIMENTAL VALIDITY INTERNAL This threatens the researchers ability to know if their study is accurate at all Placebo Effects Participants change their behavior because they are in a study Hawthorne Effect Participants change their behavior becausethey being observed are Maturation Participants behavior changes over time but  it would have even if they weren't in the study Mortality Participants drop out of the study whichcould change the end results Having a control group helps reduce these effects Observer Bias Researcher's knowledge of the research study's purpose variables and hypotheses biases their observations of the DependentVariable in some way Experimenter Effect Researcher unconsciously behaves differently towards members of the control group and the experimental group thus biasing the response of participants Researcher Attribute Effect Some characteristic of the researcher confederate systematically biases the results EXTERNAL Your results just doesn't hold with the world outside h I 1 at di th  Threats to external validity minimize the the stud your study researcher's ability to generalizebetide the results in the Also called Ecological Validity lab are infact what people would act like in the lab but it doesn't translate to the environment that people spend most of their time Somethinghappened in the lab to make the study samplereally different from the general population Testing Interaction Reactive arrangements Artificial setting of being in an experiment might affect participant results Selection Interaction The sample selected for the experiment may be uniquely differentfrom the study population History Interaction Results may not be generalizable due to time period and social when experiment took place 8 peased How To ADDRESS ECOLOGICAL VALIDITY Issues the apartment setting Use of creative procedures Field experiments 1  researcherstrytd so that when Note the limitations recreatethey can LIMITATIONS OF QUANTITATIVE RESEARCH Quantitative Concerns Ecological Validity Would the results of the study hold up outsideof the research environment Precision Is the conceptualization of the variables clear and logical Reliability Are the measures of the study internally reliable Are the indicators measuring the same thing Self Report Biases social desirability bias lack of memory knowledge perception good participant effect Confound Is there some other variable that could be theoretically driving the effect other than the  cally 9 measured variables Sampling Concerns Does the sample adequately represent the population the researcher s are trying to generalize to AN ETHICAL FRAMEWORK Human Subjects Research Human beings are involved Use of private information that can easily identify individuals Bodily materials Tree Principles Ethics boardcheck Respect for persons Individuals as autonomous agents Persons with diminished autonomy are entitled to protection other mental Youth Cognitive impairment health issues Informed Consent Must describe  Purpose Procedures Alternatives Benefits Time Incentive Statempationthatoluntary Risks Right that dentiality and right to Consent for Welfare Researchers are obligated to secure participants well being Justice Fairness in the distribution of risks benefits INFORMED CONSENT FOR ONLINE RESEARCH TECHNOLOGY RESEARCH ETHICS Technology makes large corpuses of data easily searche scraped analyzed However is this ethical Ee.it How To Ensure Participants Are Truly Informed Do participants understand algorithms No Hallinan Brubaker Fiesler 2019 Do participants read the terms of service No 0bar Deldorf Hirsh 2020 Is The Data Considered Public Argyman Yeschment section How talould People lathose Content Ended Up InThe Study ViewThe Research STRATEGIESTO MINIMIZE RISK Obtain informed consent if possible Delete names and other identifiable information Note Direct quotes fromtweets and other posts as identifiable information Attempt to acquire informed consent prior to dissemination Useful in cases where only a few people will be identifiable ASSOCIATION OF INTERNET RESEARCHERS GUIDELINES Started in 2002 3 pgs 82 pages INTERNET RESEARCH Big data from Scrapes API Building dashboards Social listening Algorithm design Social analytics User Experience  Semantic Sentiment Analysis Digital Rhetoric INTERNET RESTARCH ETHICS 3.0 Venue Platform related considerations thepectedthis offthinned th What are the ethical Is informed consent possible in this venue Ethical and Legal considerations What are the legal requirements for the researcher REB Canada in What are the ethical assumptions of the subjects What are the risks benefits of the research DATA MANAGEMENT How is the data being stored managed represented How is potentially sensitive data being secured lathat if the data includes information regarding self harm criminal activity How will the data be anonymized What are the ethical considerations of this anonymization Data minimization Collect and store no more data than is necessary to answer the research question LEGAL CONSIDERATIONS Is your data collection and use within the Terms of Service Does your data collection and use comply with local laws Does your project open up users participants to Legal risk Is the subject politically sensitive Will you be able to maintain user privacy BENEFITS Can the data the relevant question answer What do we know about the sample of data of users etc Because social media andmuch of the internet is proprietary many of the answers to important questio related to generalizability are unavailable to independent researchers WHOSE ETHICS ETHICS LEGISLATION Where did ethics legislation come from Tuskegee Syphilis Study The Stanford Prison Study Milgram's Obedience Studies BEYOND THE REB REBS provide a baseline of ethicalguidelines within a legal framework They protect participants and also the universityfrom liability REBs are tied to federal research funding Research exists outside of the university Market research User Experience Journalists Opinion Political polling Social Media Platforms These may be subject to other regulatory bodies or Codes of Conduct Or they may not Remember Just because you ian doesn't mean you shout Jurassic Park MORE ON LIMITATIONS OF QUANT RESTARCH Precision Is there slippage between the conceptualization an the operationalization Reliability Measurement Error Differencebetween a measure quantity and its true value We are very unlikely to know the true value Sampling Concerns Size Power Is the sample large enough to detect the effects the researcher is interested in Small underpoweredstudies may increase faffes But large samples can also identify very small effects that are not of substantive import Overstating The Implications Research hardly ever proves something particular in the social science where the focus is falsifiability  To say that one variable concept causes another requires meeting the criteria for causation Remember Correlation does not equal Causation Effect sizes Also Significance and Confidence Internals MORE ON ETHICS Ethics are typically not Yes No questions Ethical Judgements rely on guidelines not recipes Regulations are not all of the ethical guidelines researchers should adhere to but they are an importan starting point Informed Consent For parties that cannot give informed consent consent may be granted by an authorized trusted person a legal guardian However assent at an appropriate level should be obtained Concern For Welfare Maximize benefits and minimize harms to the individu  Risks and benefits should be outlined in the informe consent so that the participant can make their own autonomous decisions What Do REBs Look For The REB consists of a panel of experts in different area that review ethics applications prior to data collection They look for Have the risks been minimized Are they reasonabl incomparison to benefits Is the selection of subjects equitable Are there privacy confidentiality procedures What is the data monitoringplan How will informed consentbe obtained Are safeguards in place to protect vulnerable populations