Quantitative Research Methods in Political Science Lecture 2 PDF

Quantitative Research Methods in Political Science Lecture 2: Conceptualization and Operationalization Course Instructor: Michael E. Campbell Course Number: PSCI 2702(A) Date: 09/12/2024 The Wheel of Science Source:...

Quantitative Research Methods in Political Science Lecture 2: Conceptualization and Operationalization Course Instructor: Michael E. Campbell Course Number: PSCI 2702(A) Date: 09/12/2024 The Wheel of Science Source: W. Wallace. 1971. The Logic of Science in Sociology. Sourced from Healey, Donoghue, and Prus. 2023. Statistics a Tool for Social Research and Data Analysis. 5th ed.  Causality is the idea that one thing causes another thing to happen In statistics, causation means Causation and variation in one variable causes variation in another variable Causal Think of it in terms of Cause and Relationships Effect When thinking about causation, consider: 1. Covariance of Phenomena 2. Temporality 3. Causal Mechanisms/Pathways 4. Non-Spurious Covariance  Covariance and Temporality and Causal Pathways Covariance Temporality Temporality: the point in time The identification of a patterned at which the phenomena you relationship between phenomena are studying occur For instance, let’s say you have two For causation to occur, one variables each representing a thing must precede the other concept… in time These variables must vary together in Example: Smoking causes some way Cancer For example, as the values on one The act of smoking must variable increase or decrease, the precede the cancer diagnosis values on another variable must increase or decrease  Causal Mechanisms Causal mechanisms can be understood as the pathways through which an outcome occurs Can also think of it in terms of the explanatory links between variables Example: Smoking causes Cancer – but how? Carcinogens inhaled from cigarettes alter the DNA in your lung cells resulting in the production of cancer cells This is based on the theory of carcinogens Visualize all of the causal mechanisms in a game of Mouse This is why we use theoretical Trap  In lecture 1 we theorized that the more informed an electorate, the higher the level of political participation We used two variables to represent these concepts: (1) # of households with internet access; (2) voter turnout We hypothesized that in countries where more households Causal had internet access, the level of voter turnout would increase Mechanism But it is not necessarily internet access that is causing variation in the level of voter turnout… Example Instead, it might be that when more people have access to the internet, the more likely they are to have information about politicians’ policies Therefore, the causal mechanism is… More households with internet access  more information about policies provided to constituents  likelihood of voting increases Non-Spurious Covariance The presence of covariance does not necessarily mean causation Always remember, “correlation does not mean causation.” Just because two things co-occur, does not mean that one causes another For example, winter does not cause spring (even if it always precedes it in time) Therefore, always be sure that variance in one thing is the cause of variance in the other, and that it isn’t happening by random chance  Causality and Variables (Independent and Dependent) Independent Variables Dependent Variables Represented by symbol X Represented by symbol Y Variation in the The dependent variable is independent variable is hypothesized to vary hypothesized to cause because of variation in variation in the dependent the independent variable variable  Independent and Dependent Variables Cont’d Smokin Cance g r The The Independe Depende nt Variable IV will always nt (X) precede the DV in Variable (Y) Occurs at time Time 1 Occurs at Time XY 2 Confounding Variables Related to the idea of spurious covariance Example: you find covariance between high ice cream sales and more drownings Therefore, you might assume high ice cream sales cause more drownings But in reality, it is something else causing variation in both variables…  Confounding Variables Cont’d In our example, it is the temperature which is affecting variation in both variables… X Ice Cream Sales High temperature causes more people to buy ice High Z cream and to go swimming Temp. The more people swim, the more likely they are to drown Y # of Spurious Relationship: Drownings when there is seeming Therefore, when a spurious relationship exists and is controlled for, the apparent relationship association between two between X and Y disappears variables, but another variable is the true source of variation in Confounding variables are represented by symbol Z each. Therefore, we must anticipate alternative Conceptualizati on and Operationalizat ion  Conceptualization and Operationalization “Politics…is all about making choices” (Pollock III and Edwards 2020, 1). Concept: “an idea or mental construct that organizes, maps, and helps us to understand phenomena in the real world and make choices” (Pollock III and Edwards 2020, 1). In other words, concepts are abstractions… Concepts can be more or less complex – e.g., “globalization,” “power,” and “democratization” are all concepts Question: What is a political Party? (seems simple enough to answer)  Example: ‘Political Party’ as a Concept Political Thinker Definition Edmund Burke (1770) “[A] party is a body of men united, for promoting by their joint endeavors the national interest, upon some particular principle in which they are all agreed.” Anthony Downs (1957) “[A] political party is a coalition of men seeking to control the governing apparatus by legal means. By coalition, we mean a group of individuals who have certain ends in common and cooperate with each other to achieve them.” V.O. Key, Jr. (1964) “A political party, at least on the American scene, tends to be a “group” of a peculiar sort…Within the body of voters as a whole, groups are formed of persons who regard themselves as party members…In another sense, a “party” may refer to the group of more or less professional workers…At times party denotes groups within the government….Often it refers to an entity which rolls into one the party-in the-electorate, the professional group, the party-in-the-legislature, and the party-in-the-government…In truth, this all encompassing usage has its legitimate application, for all the types of groups called party interact more or less closely and at times may be as one. Yet both analytically and operationally the term ‘party’ most of the time must refer to several types of group; and it is useful to keep relatively clear the meaning in which the term is used.” William Nisbet “[A] political party in the modern sense may be thought of as a relatively durable social formation which seeks offices or power Chambers (1967) in government, exhibits a structure or organization which links leaders at the centers of government to a significant popular following in the political arena and its local enclaves, and generates in-group perspectives or at least symbols of identification or loyalty.” Leon D. Epstein (1980) “[What] is meant by political party [is] any group, however loosely organized, seeking to elect government office holders under a given label.” Ronald Reagan (1984) “A political party isn’t a fraternity. It isn’t something like the tie you wear. You band together in a political party because of certain beliefs of what government should be” Robert Huckshorn “[A] political party is an autonomous group of citizens having the purpose of making nominations and contesting elections in (1984) hope of gaining control over governmental power through the capture of public offices and the organization of the government.” Joseph Schlesinger “A political party is a group organized to gain control of government in the name of the group by winning election to public (1991) office.” Even if a concept seems simple, it can nevertheless mean many different John Aldrich (1995) “Political parties can be seen as coalitions of elites to capture and use political office. [But] a political party is more than a  Concepts Concepts can be more-or-less complex – e.g., “globalization,” “power,” and “democratization” are all broad concepts What does someone mean when they refer to a concept? It is hard to tell because concepts are ideas Example: you have a research question: “does globalization exacerbate climate change?” It is impossible to answer the question in this form, it is a conceptual question The concepts are too ambiguous for us to work with in these types of questions…  Conceptual and Concrete Questions Conceptual Question: “a question expressed using ideas, is frequently unclear and is difficult to answer empirically” (Pollock III and Edwards 2020, 2). To answer a conceptual question requires the process of conceptualization and operationalization… This will transform concepts into concrete terms so that can be described and analyzed In so doing, you can develop a concrete question Concrete Question: “a question expressed using tangible properties, [which] can be answered empirically” (Pollock III and Edwards 2020, 2).  Conceptual Definition: “Clearly describes the concept’s measurable properties and specifies the units of analysis (e.g., people, nations, states, and so on) to which the concept applies” (Pollock III and Edwards 2020, 2). They are more precise and clear expressions of Conceptual phenomena under review Definitions Can only be written once you have selected a set of properties that best represent the concept “Communicates the subjects to which the concept applies and suggests a measurement strategy” (Pollock III and Edwards 2020, 3).  Operational Definition: “describes the instrument to be used in measuring the concept and putting a conceptual definition “into operation”” (Pollock III and Edwards 2020, 8). In other words, it indicates how we know if there is more or less of a phenomenon Operational Definitions An operational definition will “describe explicitly how a concept is to measured empirically” (Pollock III and Edwards 2020, 10). Example: you are measuring height. Operational definition: “Height” is defined by the number of feet/inches a person is tall.”  The Process of Conceptualization and Operationalization Step 1 Step 2 Step 3 Step 4 Clarify the Concept Develop Conceptual Operationalization Compute or Select Definition Variable Select essential Communicate Develop operational Gather data or select characteristic(s) necessary definitional definition relevant variable(s) components Develop instrumentation  The first step of clarification is to identify the concepts concrete properties – i.e., a list of properties that best represent the concept Step 1: Properties have two characteristics: Clarify the 1. They must be concrete (i.e., they must be perceptible, meaning we must be able to observe them) Concept 2. They must vary (i.e., they must occur or not occur, or occur at different levels) Example: Does globalization exacerbate climate change? Begin by identifying the concepts in the conceptual question In this case: (1) globalization; and (2) climate change  Example: Conceptual Clarification of “Globalization” Globalization has certain characteristics associated with it (a general understanding exists) You must identify these essential characteristics – which must be concrete and variable Identifying these characteristics will reduce conceptual ambiguity Can be identified using intuition or through research (use research in assignments!) Conceptual Clarification of “Globalization” Cont’d When you identify the characteristics, you then narrow them down to identify the most essential characteristic(s) International Monetary Fund (IMF) identifies four characteristics of “Globalization”: 1. Trade and Transactions 2. Capital Investment and Movement 3. Migration and the Movement of People 4. The dissemination of Knowledge Each of these are tangible properties, because we can perceive them (and measure them directly) Globalization – Essential Characteristics  Not only do we need to select a characteristic that best represents the concept, but it must be consistent with the context of our research. And the data available to us. Selection requires that we make a value judgement! While multidimensional concepts exist, “as much as possible, you should define your concepts in clear, unidimensional terms” Globalization – Essential Characteristics (Pollock III and Edwards 2020, 7). Cont’d  Justification for trade and transactions: they are legally binding and necessarily tie different countries closer together. We might then argue that higher levels of trade represent higher levels of transnational integration. Simultaneously, there may be justifications why the other characteristics may not represent our concept as well as trade… Globalization – Essential Characteristics Cont’d  Capital and investment movements can increase without the presence of global institutions or regulations that guide and harmonize these activities on a global scale Globalization – Essential Characteristics Cont’d  Migration of people across borders occurs within specific regions or between neighboring countries (e.g., in the European Union or in South America), but they can increase without impacting global migration trends. Globalization – Essential Characteristics Cont’d  Knowledge dissemination can increase without being global if access is limited to certain regions, countries, or socio- economic groups. For instance, digital divides and disparities in education mean that increased knowledge flow might not be global in Globalization – Essential Characteristics scope. Cont’d Globalization – Essential Characteristics Cont’d  Step 2: Develop Conceptual Definition Once you have identified your concept’s essential characteristic, you need to develop a conceptual definition Conceptual Definition Template (See: Pollock III and Edwards 2020, A conceptual definition must 7): communicate three things: 1. The variation within a measurable characteristic (or “The concept of ________ is defined as set of characteristics) the extent to which ________ exhibit the characteristics of ________.” 2. The subject or groups to which the concept applies (unit of analysis) 3. How the characteristic is to be measured  The concept of (A) globalization is defined as Step 2 - the extent to which (B) countries exhibit the characteristics of (C) high levels of trade Develop and transactions Conceptual Components represent: Definition A. By stating that (A) globalization is defined by the extent to which, it restates the name of the broad idea Cont’d being conceptualized, but it also points to the existence of variation, meaning that it can exist in varying levels, or not at all B. By identifying (B) countries, we are identifying the subjects to whom the concept applies C. By stating (C) high levels of trade and transactions, we’re identifying the way that the concept can be measured  The unit of analysis is the entity to which your concept applies, and the entity we want to describe and analyze There are two levels of analysis: 1. The Individual Level of Analysis Is oriented towards the study of individual political Unit of behavior At this level, we can learn something about individual preferences and attitudes about something in particular Analysis 2. The Aggregate Level of Analysis: Is a collection of individual entities For instance, countries are studied at the aggregate level, because they are aggregations of multiple entities Warning: be careful when applying aggregate level results to make inferences at the individual level – it risks an Ecological Fallacy  Step 3: Develop Operational Definition (Operationalization) Operational definitions are extensions of conceptual definitions that explicitly state how the concept will be measured empirically For example, how would we know if trade was occurring at a high levels among countries? Example operational definition of Trade and Transactions: the total amount of imports and exports of a country, in millions of current year US dollars Operational definitions are almost always accompanied by an instrument Instruments are tools used to measure a specific concept, variable, or phenomenon in a systematic and reliable way  Instrument Example Please respond to the following. On a scale of 1 to 10, with 10 being “Very High,” and 1 being “Very Low,” rate your shopping experience Instruments can be simple or complex depending at Sephora… on complexity of characteristic you are measuring 1 – Very Low 2 Example: You want to know peoples’ satisfaction 3 with shopping experience at Sephora 4 5 Operational definition: The shopping 6 experience at Sephora is defined as the overall 7 satisfaction of customers during their visit to the 8 store 9 10 – Very High To the right is an instrument that will measure ordinal level data  Measurement Error It is important that instruments measure intended characteristics, as opposed to unintended characteristics You want to maximize the fit between the definition of the concept and the empirical measure of the concept Otherwise, your data may contain 1. Systematic Measurement Error (a.k.a. Measurement Bias) 2. Random Measurement Error These types of error disrupt the link between the concept and the empirical measure Systematic Measurement Error Systematic Measurement Error: “introduces consistent, chronic distortion into an empirical measurement…[by producing] Imagine if the above instrument added operational readings that consistently mismeasure the characteristic the or omitted a key trait. researcher is after” (Pollock III and Edwards 2020, 13). For example, what if it measured only the imports of a country and not the Occurs when instrument captures exports. unwanted traits, or omits necessary traits that artificially inflate or deflate It would provide data on trade that were what you are trying to measure consistently lower every time it was used. There would be inconsistencies between the observed and true values of trade.  Systematic Measurement Error Visualized True mean value Observed mean with properly value of trade if defined instrument exports are that accounts for omitted from imports and instrument exports The presence of systematic measurement error will create bias in estimation on every single occasion because error is built into  Is the result of random chance. Random measurement error introduces “haphazard chaotic distortion into an empirical measurement, producing inconsistent operational readings of a concept” (Pollock III and Edwards Random 2020, 13). Measureme Even if instrument is free from systematic error, nt Error there is a random chance the measurement will be wrong Example: you are measuring trade in a country whose main export is lumber, but there was a significant lumber shortage that year (resulting in lower observed values) Resolve by looking at mean value over time  Validity and Reliability Reliability The validity ofValidity a measure is “the extent to which it records the true The reliability of a measure is “the extent value of the intended characteristic to which it is a consistent measure of a concept” (Pollock III and Edwards 2020, and does not measure unintended 16). characteristics” (Pollock III and Edwards 2020, 17). To be reliable, the measurement should give the same reading every time Even if reliability is lacking, we can assume that if we used the measure A measure can still have systematic error enough over time that the problem of and be reliable, but results must be random error would correct itself and consistent reflect the true value of the characteristic As random error increases, reliability is negatively affected because the results Thus, we can say validity is more will be more inconsistent important than reliability (but both are still important) Reliability and Validity Visualized Validity and Reliability Visualized Cont’d Reliable Results consistent Results consistent but do not across contexts and accurately the measure represent concept accurately Invalid Valid represents concept Results Results inconsistent inconsistent across contexts but and do not measure accurately accurately represents concept measure concept Unreliable  Step 4: Variable Selection Example: Once you have your operational Does Globalization exacerbate Climate definition and instrument, you can begin Change? to collect data Globalization conceptualized as Trade Operationalized to measure countries’ The result will be a variable that imports / exports Therefore, I can select the following variable represents the characteristic(s) of the concept you were trying to measure If you are not collecting the data yourself, though, you can select a variable that best reflects your conceptual and operational definitions Source: Quality of Government (2022)