Unit 5 Lecture Notes.docx

Unit 5 Lecture Notes Approaches to Knowledge This next series of videos will be examining and talking about research methods, which I will immediately confess upfront is not one of my favorite topics. However, I would have to say, among all the things I've learned in sociology, this to me feels like...

Unit 5 Lecture Notes Approaches to Knowledge This next series of videos will be examining and talking about research methods, which I will immediately confess upfront is not one of my favorite topics. However, I would have to say, among all the things I've learned in sociology, this to me feels like a very, very valuable piece because understanding which research out there is legitimate. Because I now understand how to put things together, how to try to stay as objective as possible, how to examine who paid for the study, how large their study group was, all effects, what you can and cannot say about the results. So, when I'm looking at McLain's or the Winnipeg Free Press or a study posted online somewhere, I know how to assess that, at least to some degree. Let’s start by talking with the scientific approach. What's probably important to know about the scientific approach, is that it's an approach. It's a methodology. It's a way of collecting knowledge. It's a process. And that process produces content. In sociology and psychology, in chemistry and biology, it produces contents. Overall, the scientific approach is a process by which you collect knowledge. So that's how we approach knowledge. And here is an interesting word/concept: epistemology. How do we know what we know? So, I'm going to ask you some questions: What do you think, is the world is flat or around? Yes, or no? And how do you know what you know about that? I'm going to assume that most of you answered yes to that question. And how did you know that? Do you know that because you have a lived experience of the earth is round. Do you not experience the world whether you're walking or cycling or driving or running as flat? You don't experience it as round and yet you believe it to be round. And why is that? How do you know that then if you don't know it from your own personal observations. I would assume that you know it because people with the authority to teach it that you've granted that authority talk to you that it's round. And they showed you pictures of a world that's round. Even though currently, almost anything can be faked in terms of a picture or a recording, for that matter. But trusting the people that you were with, who taught you that, that's how you know what you know about the earth. The second question you were asked to think about is, do you think being kind makes you a more likable person? Again? I'm going to assume that you probably answered that. Yes. And so how did you know that? How do you know what you know about that? Probably because you've had experiences. That being kind has produced positive results for you. Kindness begets kindness. You are liked for that. It may have been taught to you by your parents or other significant figures in your life as well. So different ways of knowing whatever it is that, you know, in that kind of a context, how do we know the things that we know? Well, here's sort of a list: intuition, quick ready, insight based on feelings and unknown inner sources. So, someone that you live with and know well comes home at the end of the day and says one sentence and you take one look at them, and you just know something's wrong. You don't know what it is. You don't know why, but you already get a read from them that everything is not well. If I said to you, can you explain to me what it was you were observing that had you knowing that you probably wouldn't be able to tell me. You just knew. You intuited it based on your feelings. That's a way of knowing intuition. Common sense is another way of knowing opinions that are widely held because they seem so obviously correct. Don't throw the baby out with the bathwater. Of course not. Just because you don't like one thing doesn't mean that you should throw out the whole of something because there might be other parts of that that are useful or that you do like. Many hands make light work. That's just common sense. But sometimes common sense gets a little muddled. Birds of a feather flock together. Yeah. People who have things in common like to be together. But it's also said that opposites attract people who are different from each other, form friendships or relationships of some sort based on difference. Well, that directly contradicts what I just said. So common sense isn't quite as common sense as we might always think. Another one is knowing tradition. This falls directly in the world of culture that which is transmitted from one generation to the next. So historically, we have, based on tradition, believed that women are more natural parents than men because they have birthed the baby. Lots of people I would, I think, still would say that that's true. And how do they know that? Because of their observations. And yet what we're seeing now is a significant rise in fathering and fathering itself is a new word. We have always said mothering, but fathering is a new word, which indicates to us that there is some kind of social change afoot because now we have a new word for it. Turns out there are many, many, many men who are very natural nurturers, who are very interested, who really enjoy it. And they are very good at fathering. And so, it turns out that our traditional belief about this isn't quite accurate based on what's happening in the lived world. But we have believed it and internalized it because that has been our tradition. You shouldn't eat dog meat like yuck, not squirrels. We just know that. And yet in another culture, they just know that you shouldn't eat beef, or they just know that you shouldn't eat pork. Their emotional response to the thought of eating pork is no different than our emotional response to the thought of eating dog meat. So, traditions are the things that everybody just knows that are traditions of your culture. They have been transmitted to you. Another approach to knowledge is authority, and this comes to us from those who are defined as qualified to produce that knowledge. So, when you trust your science teacher about the earth being round because after all, she's a science teacher, he must know what he's talking about. He's my science teacher. We're much more likely to believe a medical doctor about the fact that kissing can transmit the common cold than we would you know, Uncle Pete. In the world of smaller children, you know, "my mother/father told me" is also a form of authority. Not that much of what we think we know about the world we know by agreement. Therefore, authority is a big category where much of what we know falls into that revelation. Another way of knowing is revelation (mystical). For example, pondering if there is life after death relies on supernatural, knowledgeable authorities in your context and you will have said, I know what I know, because my sacred text, whether that's the biblical text, the Torah, the Koran, whatever that might be, tells me something about this, or my priest, my rabbi, my pastor taught me about this book that's relying on that which is mystical as your source of knowing what you know. Rationality is another way of knowing. And this is just adhering to basic logic. You know, if you are under heavy assignment pressure and you stay up late too many nights or when you eat poorly, there's a pretty good chance you're going to get sick. It's just logical that something like that is going to happen. Now there's a point at which logic doesn't work either because logic alone would have me 100% believing that the Earth is flat because I could walk and walk and walk and walk and it would still be just as flat as where I started unless I proposed to walk around. But then I'd also have to do some swimming. So, I can't walk around the earth anyway. Do you get my point? Rationality is very useful, a good way of knowing. But it eventually also can produce inaccurate results. And then we have science. This is adhering to basic logic, but adding to that observation, and that's what we would call science, empirical knowledge that must be logically valid and then empirically verified. In other words, you've tested it to make sure it's true. The other piece of science that's so important in our cultural context is that it's public and therefore is supposed to function as self-correcting. Now, it doesn't always do that perfectly, but it is set up for it to do that. So you submit something for publication and it's going to be reviewed by other peers in your discipline who might say, Yeah, you can't say this or that source isn't legit, or there's a flaw in your reasoning in the way you set up your study. Who knows what they might say to you? And then even if it gets through your peer review committee and it gets published and it's out there in a journal and then someone else launches a study like yours and gets very different results, and then your study is challenged. So again, challenged by your peers, people are checking and rechecking and the public nature of it, the consensus-based nature of the scientific approach means that more often it is closer to what is the truth then? Not because of its rigorous process of knowledge production, but its public and its communal character protects it. And now I'm back to that concept of inter-subjectivity. We're all still subjective humans. None of us are totally objective, but we have set up an approach that helps us to be more and more objective as much as we are able. And of course, you must be honest with the work that you're doing. Just because you see something with your own eyes doesn't always mean that it's accurate either. So, if you look over all these different ways of knowing, this is the list of what I've just talked our way through: Strengths and Limitations of the Scientific Method How many of these ways of knowing are actually tested in some way? Only science is routinely and habitually tested, and that's why science has gained so much power in our cultural context. Now, that doesn’t mean that science sometimes doesn’t get it wrong, because scientists are still humans, but its process is more rigorous and more intentional, hence in some ways more reliable than some of these other forms of knowing. But it's not that science isn't also limited. Science cannot speak to the metaphors. That last question you were asked to think about is there life after death? While science has no business answering that question because it's not something that we have access to observing. It cannot be empirically tested. Science can make no comment on that within its own methodological assumptions. There are also some common errors in personal human inquiry. This is just important to think about. Now, I want you to pause and look at this picture for a bit, because I think this picture is a good metaphor for what I'm about to talk about. But if you look at this picture, you can see that this is a mirror, a person's looking in the mirror and this should be an accurate reflection of herself, himself in the mirror. But the mirror is still cracked, shattered and in pieces. And so, it does distort. So, yeah, we're getting an approximate sense of how this individual looks, but not totally, because the cracks and the shards are distorted. And think about that as you think about the world of human inquiry. So here are some common errors in accurate description: Inaccurate observation Have you ever hung out with a bunch of people, and something happened and then, you know, a month later, a year later, a week later, you talk about it, and someone remembers it differently than you remembered, and the next person remembers it slightly differently. Right. Even though you're all there at the same time and you all saw and experienced the same thing, our memories still lock in on different things. And over time, our remembering of our remembering can sometimes shift our remembering. So not all our observations are always entirely accurate. over generalizing. So, I have observed such and such about pregnant women. I've known probably about 40 pregnant women in my life. I can't generalize that to all pregnant women because there's millions of pregnant women in all kinds of different cultures and my observation of 40 can't be generalized to all those women because my sample size of observation is much too small. All I can say is all the women I know who have been pregnant, this is what I have observed. That is a very fair statement to make. But when we overgeneralize, usually error occurs. Selective observation. It's easy to accept the facts that support our argument and ignore the facts that counter argument. We're all human. We all know we've done this. I don't want to read that. I don't hear about that because that doesn't support what I think. And I want to think what I think. And I don't want to be made uncomfortable by a set of ideas that might challenge what I think. So, I selectively observe. Fabricated information Sometimes we just make stuff up, sometimes others just make stuff up. That happens. Illogical reasoning. The poker player who's lost, you know, seven rounds in a row and then says, okay, I'm going to play one more round because I'm bound to win because I've already lost so many times. That's illogical reasoning, but we all do it in one way or another. Ego involvement in our understanding. If I've committed to something, I’ve spoken out about it. I've held a position. I've taken a position. My ego is invested in that position. Remaining and being correct. New information that comes to me indicates that I'm going to have to change my position. That I could be wrong. Well, that's tricky. So, my ego is involved in my understanding, and it might skew how I perceive and what I'm willing to hear. Premature closure of inquiry Or I might just say, “That’s enough. I don't want to read anything more about this. I've read enough. I know what I know. I don't want any more new information." And you can hear that theme coming up in a number of these, even though they are slightly different angles on the same thing. Mystification You know, something deep inside of me just moved me to understand that this is the choice I should make. And that's not really a thing we're going to challenge in someone else either, you know, no matter what religious system you're in. I prayed about it, I meditated about it. It came to me in a dream. As soon as we mystify something that we're going to put forward as a justification for what we think or something we're about to do. That can't really be touched by other people. What I'm suggesting is that people who are onto this can also use this to justify something and that can create common errors. Assumptions about Science Okay, some assumptions about science and this is science at large. I'll talk more specifically about sociology in just a minute, but these are some of the assumptions of science as an approach to gathering knowledge. First, the assumption is that nature is orderly. That an apple will never fall up. It will always fall. There is an order to nature. That we can know nature We can study it; we can figure it out. We can know stuff. Science also assumes that knowledge is always superior to ignorance. Now, I think in many contexts I've been challenged. Like, if you think about Oppenheimer, who developed the atomic bomb and then who later expressed his regret, he no longer thought that knowledge was superior to ignorance. He lived with regret about that. And that's a really complicated one ethically, because what if we hadn't had that atomic bomb? How would that WWII have turned out and the impact it has on the overall fabric of war? I am not commenting on the ethics. I'm simply commenting on the fact that in some cases, maybe knowledge isn't superior to ignorance, but science assumes it always is. Science also assumes that natural phenomena have natural causes. Which is why science cannot study the world beyond our tactile physical world. It cannot study the afterlife. It could study the effects of believing in the afterlife like sociology does. The effects on the individual of believing that. But it can't study the actual afterlife because it assumes natural causes. Science also assumes that nothing is self-evident. Just because it looks that way, seems logical. Common sense. However, this is not enough in science; it still needs to be tested. And that knowledge is derived from the acquisition of experience. It's empirical. You must set up a study. You must measure things before you can really know what you know. These are all assumptions of science. What Are the Aims of Social Science? First, we want to explore it. We start with our questions. Remember, sociology is about asking questions and then our smaller and bigger questions eventually translate into smaller and bigger ideas. Little light bulb moments, medium light bulb moments, and some big light bulb moments as we explore and develop our questions. So, we start with an initial rough understanding of something. We look around and let's say we see that men seem to be more competitive than women. That's what we want to explore. Second, remember we talked about this; we want to get a good description. What can we know about a statement like that? What kinds of questions can we ask? What kinds of surveys can we do? What kinds of observations can we set up? How can we get a good description, precise measurements and reporting of characteristics around this? Well, we can use surveys, focus groups, and individual interviews, and we do all those things. And we bring back together the data that we've gathered, and we find that women are also competitive, but they're competitive in different areas then men are competitive. Well, the results show that men might be more competitive about sport, but women might be more competitive about appearance. In both cases, cultures have actually taught that those are part of the gender stereotypes we embed in what we teach boys and girls about being male and female. But the more that we dig around and the more our questions prompt better questions that we keep, you know, collecting data, we find that women are becoming increasingly competitive in initially traditional male areas of competition in the world of sport were. And we find that men are becoming more competitive in the world of appearance, fashion, personal hygiene, etc., etc. And I'm summarizing real results here. These are the kinds of things we find when we go and try to get a good description of what's going on. And then we want to interpret/explain those observations like, why? Why do men suddenly care about their appearance and how they dress when historically, you know, a couple of decades back they didn't to nearly the same degree that it appears they do now? What's going on here? Well, that's when we get to the theorizing piece. So, is it true that the capitalist market has sort of tapped out all the kinds of pressures they can put on women to change their appearance? I mean, we're all the way to, you know, genital surgeries now in terms of making women feel like they need to change their bodies to fit certain beauty standards. And so, a capitalist market that wants to keep making money has now maybe turned its attention to men. What can we convince men about that we could sell them so that we could make some money off men. So, we're now targeting men in a way that we used to target women. And then, of course, we start predicting. So now that we know this, what other things do we think might be happening? What else is going to start happening? Well, if there was more pressure on women to surgically change their bodies with the Botox and the breast augmentation and the general surgeries, we will now predict that men are going to start to feel like they need to do penis enlargement surgeries, which actually is starting to happen, that maybe they also have to shave their chests, which by now is starting to happen, etc. And in the process, we gain a greater understanding of that statement. It appears that men are more competitive than women, and that statement doesn’t stand. It's much more nuanced, it’s much more complex. And going through this process that I've just outlined for you gives us all kinds of interesting information about the human experience related to that topic. So back to this picture. Right. We've got questions. We've got a question. We get a little light bulb answer, we toss it back, we get another question, we toss it back. We got another that's a bit of a bigger answer, a more thorough explanation. Back to another question. And then that question becomes another light bulb moment and an even bigger one. And that's how we develop the body of knowledge that we do in the world of sociology. The Nature of Causation We're continuing our conversation about research methods. And in this case, we're going to start with discussing the nature of causation. So, A, causes B, does it, or doesn't it? That's the question. How do we know for sure that we can confidently say that yes, A causes B. The natural sciences are what we would call the deterministic sciences. And I've already talked about this earlier on in some of the videos. The idea that when you put a certain group of chemicals together, you're always going to get the same result if the increments, the amounts of the chemicals are the same. That's not how it works in the world of studying humans, you can do the same thing a second or third or fourth time and get a slightly different result every time, even if you're studying the same group of people. Because humans adjust, interpret, and change their behavior from one time to the next. So, we can't talk about sociology as a deterministic science. What we can do is talk about sociology as a probabilistic science. So, what that means is it's an attempt to isolate the few most important factors. You know, what are the biggest things going on here that provide a partial, hopefully a significantly large but partial explanation of whatever social phenomenon we're seeing here of the behavior of many people, not just a few, but a large group of people. That's sociology. That's typically the best we can do for the nature of causation because we're not studying atoms and chemicals. We are studying humans. Types of Causes So, what are the different types of causes? Well, we've got a few couple of categories of those causes or factors. The first is what we would call a contributing factor. Let's take the very oversimplified example of what needs to happen for someone to gain weight. So, in what way will A cause B? How will weight be gained? Well, obviously, if you don't exercise or aren’t very active, there's a greater chance you could gain weight. That's a contributing factor. If you make poor food choices, too much McDonald's, not enough vegetables that could contribute, or if you do a lot of binge eating, you consume a lot of calories consistently that can cause weight gain. What if there is something about your genetics, your DNA, your family history? That means your body hangs onto weight more than the next person who eats the same amount as you do. Or what if you have a disease that is making your digestive system or some part of your body malfunction in a way that also hangs on to weight? Those are all contributing factors to weight gain. Necessary Factors This is what's absolutely necessary to weight gain. Now, of all the things that I just listed that are contributing, the only one you absolutely must have is food. You must eat. You must consume calories if you want to gain weight. That is an entirely necessary factor. The other ones are not. We also talk about sufficient factors. Is there anything that will be absolutely sufficient to guarantee that there will be weight gain? In the example I'm using, there isn't. Because some people can eat just as much as the next person and not gain any weight. So, food isn't a sufficient factor. It's a necessary one, but it's not a sufficient one. In the world of the social sciences, it's very rare for us to find something that's a sufficient cause for explaining any social phenomenon. Sometimes we can explain a necessary factor. We can prove that. But what we most often find is contributing factors that this, this, and this contributes to that social phenomenon. That's usually what we get. Criteria for Causality Now we're talking about the nature of causation, right? A causes B, what's the criteria for us to be able to make that statement confidently? What's the criteria for proving that there is actual cause of some sort? Well, the time sequence must be correct. I'm going to use another oversimplified example just so we can grasp the concept wherever there are fires, there are fire trucks. Therefore, fire trucks cause fires. And I don't have the time sequence correct there, do I? Though the two things do clearly correlate, and this is an important word. They correlate wherever fire trucks are high in number, fires are high in number. But I still must get that time sequence correct. So, if I'm speculating about cause and I see that there's a correlation, I must recognize that first there's a fire and then there's a fire truck. It's not the other way around. There's a fire truck and then there's a fire. So, you must pay attention to time sequence. Now, in my very oversimplified example, it's obvious, but in a lot of the things we study in the social world, it isn't nearly that obvious. Correlation between two variables has to be present. A in some way affects B. So, whenever A goes up, B goes up or whenever A goes up, B goes down. That's also a correlation. Or whenever A goes down, B goes down. That's also a correlation if they both move at the same time so they can either move up together or down together or in opposite directions together. The point is that whenever A moves, so does B, That's a correlation. They're related to each other in some way. Correlation must not be explainable by a third variable. And then, of course, we want to start asking more questions so that we can understand the nature of that correlation. But the third important thing is that the correlation can't be explained by a third variable, which we would then cause a spurious or a false variable. Again, I'm going to take just an oversimplified example so we can catch the concept. Ice cream sales correlate with drownings. When ice cream sales go up, we see that the number of drownings also goes up. There's a very strong correlation in a similar direction. In the same direction. Ice cream sales go up, drownings go up. Now you're already going, though. That's ridiculous. Ice cream sales clearly do not cause drownings because we're using some logic to assess that and saying that just flat out doesn't make sense. So, what would be a better set of A and B? Well, temperature would get us closer, wouldn't it? Because temperature means the water is warm and people are swimming. So, it's a more logical correlation to say that when the temperature goes up. In other words, when its summer, drowning rates go up. That is a logical connection, isn't it? And temperature affects both ice cream sales and drownings. So, it explains both those variables. So, we would observe that ice cream sales as a spurious or false variable. It logically doesn't even make any sense. Again, this is an oversimplified example, but let's plunk something else in here that's more complex. Overcrowding causes delinquency. Does that seem like a fair cause and effect? Well, if we think about that for a little while, we might conclude that poverty causes overcrowding. Because you can't afford to live in spaces that have a larger square footage. And so, there's a prior cause. It's poverty causes both overcrowding and delinquency. Again, here we would say that the spurious variable is overcrowding. Now, this gets much more nuanced and much more complex. The deeper you move in to any study that you're doing. And sometimes all that happens as a study reveals that you've been using a spurious variable. And that's valuable, too, because that means you can relaunch with a more accurate understanding because you've seen that you've revealed a spurious variable. But it's always important. And this is why it's good to have peers have their eyes on your work as well as you're developing anything where you're hoping to access or understand the degree of causality. If something is to have other people looking at your work at work as well, and helping to assess whether you are indeed using variables that will tell you something meaningful about any particular social phenomenon. The Principles of Science Going to continue our discussions about research methods. And here we're now at the place to talk about some of the principles of science. Variables When we are conducting any kind of research and a basic stock concept in any science, social or hard or otherwise, would be: variables. And variables are the characteristics of objects, people. or groups of people that can be measured in some way. So, let's take, for example, that I would like to measure the habits of first year university students. So, some of the variables, what would those be? Well, do you like to study in a group? Do you like to study alone? Are there some students who memorize? Are there some students who just read things over and over? Some students might simply rewrite their notes as a way of lodging them in memory. Do you use cue cards when you're studying? Do you work at home in your in a private space? Are you at the local coffee shop? Do you go to the university library? Those are all variables of the habits of first year university students. And any sort of phenomenon that you might be studying and habits of first year university students is a social phenomenon. So, any time you're measuring, there's lots of variables to take into consideration. And then, of course, I can compare those variables based on your demographics. Is there a difference in gender? Well, I see different patterns among males versus females. Will I see a different pattern based on your age? Do those who are older study at the library and those who are younger study in the coffee shop or vice versa? Ethnicity, socioeconomic status. Those are all different ways in which I can measure. Another example of, let's say something I want to study would be smokers. Say, I just want to study the various things I can know about smokers. Some of the variables would be nicotine smokers, marijuana smokers, heavy smokers, social smokers. Again, comparing based on age, gender, socioeconomic status, etc. Those are all variables. And we break variables down into a couple of categories. And the first one is the dependent variable. So that's the one I want to explain. So, in this case, the dependent variable in my example would be the habits of first year university students. That's the one I want, explained the independent variable, and I'll explain that. The independent variable causes change in the dependent variable. So, the independent variables would be all those things I listed previously as variables. Do you memorize? You just read over. Do you use cue cards? Do you? Because those would cause changes. So, let's say we're looking at lung cancer. I want to examine lung cancer. Okay. That's my dependent variable. My independent variable then is going to be smoker versus nonsmoker. What are the changes that happen depending on whether you're a smoker or a nonsmoker? So, I want to see if you're a smoker. What are the lung cancer rates? At what age do you get that cancer, etc. If you're a nonsmoker, what are the rates of lung cancer? At what age do you get that cancer and other ways? I could ask that question. So, a smoker versus nonsmoker changes as the dependent variable results. Smoking changes the results, nonsmoking changes the results. Now the third thing here is the control group, because if I'm trying to measure smoking versus nonsmoking, what are other potential alternate causes of lung cancer? As we know, cancer has a genetic factor as well. So, I want to remove anyone from my study whose family already has a history of cancer because I just want to know the effects of smoking versus nonsmoking. I will also want to remove from my study anyone who's live with secondhand smoke because right now I just want to know the difference between if you smoke and if you don't smoke. So, someone who lives in secondhand smoke is a nonsmoker but is still more likely to get lung cancer. So this is my control group. I'm thinking very carefully through what other things might cause lung cancer that I want to remove, because right now all I want to know is the difference between smoking and nonsmoking. So, these are the things that I keep the same in my study that no one has a history of cancer in my study and that no one lived the secondhand smoke to any significant degree in my study. So those are the factors I'm controlling because ignoring them can greatly affect the accuracy of results. I'm leaving those folks out of my study. And then, of course, there's the hypothesis. I am going to hypothesize what I think the link is. Now, I'm using an obvious example here, just so we can grasp the concept, but I'm going to hypothesize that people who smoke have higher rates of lung cancer. And if by measuring the degree of their smoking, I'm also going to hypothesize that the more you smoke, the greater the rates of lung cancer will be. That's my hypothesis in this case. Now, I'm giving you an example where we already know what the results of these are. Often in the world of social research, we don't know. We may be reading other studies that give us good information, but we're launching a new question unless we're replicating someone else's study, which is also a good and worthy thing to do. But this is how we develop our hypothesis and then shape how we run our study or experiment. The other thing we must think about in the principles of science is measurement. My results are going to rely very heavily on the quality of my measurement. And there are two things here that we care about quite a bit, and one is validity/accuracy. The extent to which the study or the research instrument that I'm using is measuring what it's supposed to be measuring. How do you know that you're measuring what you think you're measuring? So let's say I want to launch a study and I want to look at suicide rates and how that's affected by religiosity. So, what kind of a correlation might I find? Does high religiosity reduce suicide rates, or does it increase suicide rates, or does it seem to have no effect at all, that there isn't a correlation? Let's say that's what I want to know. Well, how am I going to measure religiosity? Because I can't just survey the public because right now what I want is people who are religious adherents to their religious system. So how am I going to measure that? Well, a typical way of measuring that in the world of sociology of religion would be to ask, how often do you attend your synagogue, your mosque, or your church? So, the rate of your attendance is now we need to talk about that. Is that an accurate, fair measure of religiosity or do we also want to add a second measure? Do we want to say, and how often do you practice any kind of religious habit or ritual at home? How often do you meditate or pray or whatever it is that your religion asks you to do if you're devout? Because that would also be a measure of religiosity. If I just say, if anyone who claims to be an adherent or to have belief in a particular religious system will qualify for my study, then I could be including someone who actually never shows up and for who are adhering to the rituals of that religion don't have much meaning, but it's what they grew up in, and so they feel some kind of loose association to it. Well, am I really measuring what I'm measuring? If I am what I think I'm measuring, if I'm including those folks in my study, probably not. So, validity. Am I measuring what I think I'm measuring? And the second important concept is reliability. The extent to which a study or research instruments will continue to yield consistent results. So, if you do a study and you get these amazing results, but everybody else who studies that same thing gets very different results, then your study probably isn't very reliable. So how do you make sure that you're setting up your study so that if someone else replicates your study, that you will reliably get the same results? So, when the measurement is applied repeatedly to the same object, we yield the same results. Now, here's a good visual that helps us to understand that. So, if you look at bull's eye, number one here. Let's say those are how many we got there. We have seven different studies on, let's say, the topic of the relationship between suicide and religiosity. What we see there then is that in this study, all seven of those studies had the same result. So, it was very reliable. But it turns out that if you look at the methodology, they weren't really measuring what they thought they were measuring. They didn't have a good definition or screening for what qualified as being religious. So, it's reliable but it’s not all that valid. So, moving to bull's eye number two. Okay, so here we've got well, however many dots we have that so many studies we have there and we see that we're not near the bull's eye with any of them, just as we weren't in bull's eye screen one, we're not anywhere near. These studies are also scattered all over the place. So in this case, it's clearly not reliable because they're not lining up in the same spot. And it's also not valid because we're not getting anywhere near the bull's eye. Lastly, when a study is both reliable and valid, we know it's doing a good job of measuring the thing we're trying to measure and know. No matter how many other people watch similar studies, we seem to get the same results. And that's what we see here in Bull's Eye number three. That means it's both reliable and valid. And both those things are important in the world of social scientific research. Sampling and Sampling Design The other thing that's important as we look at the principles of science, would be how do you draw the sample of the people that will be in your study and what's your sampling design? So sampling is a small number of cases used to make inferences about all cases. So, when you want to find out the general opinion of Canadians on something, let's say whether or not we should institute a National Grandparent's Day, let's say that's the thing you want to know. And obviously you can't afford to ask everyone in the whole country that question. How do you get a sampling across the country that is a fair representation of all Canadians? Well, you must sample it, don't you? If I just go to the local library in every city and I stand outside with my survey board and my question and I ask that question to all those people, is that representative of all Canadians? No, it isn't. Because possibly a certain kind of demographic goes to the library? So, what I do know is those people who typically use library resources have this opinion, but I can't generalize that to all Canadians. What about if I stood at the corner of, let's say, Portage in Main in Winnipeg and asked all those folks? Can I generalize that to all Canadians? Well, probably not. Chances are that I will be asking mostly urban dwellers in a very particular area of the city, and that doesn't in any way represent opinions of anyone who lives outside of cities who dwells rurally. So, I can't generalize that to all Canadians either. Let's say that I go and sit in a medical clinic and ask my survey question there. You get my point, right? Obviously, I'm only going to get those who are frequenting medical clinics, which is typically the older and the younger. So that also can't be generalized to all of Canada. So, you keep hearing me use this word representative. And that is the main issue. Can you draw your sample from your study in a way that's representative of the body of people from which you're trying to get that opinion or behavior, whatever the nature of your study is? So, we talk about probability sampling. Probability sampling is a randomized sample. You can specify the probability of the unit that's being selected. So that requires accessibility to all possible units. So, you might use a computer, you might put in all possible units of the entire Canadian population and then use a computer program to draw a random sample from that. You might use a table of random numbers. You might just say every 50th Canadian is going to get a survey. It is necessary for representation, that's how you can estimate the accuracy and the representativeness of your sample. It avoids all conscious or unconscious bias of the researcher. It includes simple random sampling, systematic sampling, stratified random sampling, cluster random sampling. There's lots of different types. Some studies also use nonprobability sampling. And if you are transparent about the fact that you're doing that, that's okay too. And whether we feel we can do just nonprobability instead of probability often depends on the nature of what we're studying. But nonprobability sampling means there's no way of specifying the probability of the unit being selected. Now these are often convenient samples, so standing outside of the library, they're also cheaper because you don't have to go through the process of accessing an entire, you know, the whole list of all the Canadian population and all the time and energy it would take to do that. You may also use quarter samples or snowball samples. Or purpose of sampling. And I won't go into the details, but there's lots of different ways to do nonprobability sampling as well. And depending again on your study or how far you want to generalize your results, these are perfectly good studies, but you always must be attentive to what can you say about your results? Who can you generalize it to? How broadly can you generalize your results? That depends entirely on your sampling and your sampling design. So, let's say I wanted to take a random sample of the University of Manitoba students. I want to ask a survey question. So that's probability sampling. I want to take a random sample from all of them. Well, if I'm just going to ask the students, I already know that that's obviously not a probability sampling. Right. That's not going to be representative. If I ask all the people in the classes that I teach, well, that's not representative. There's a multitude of different disciplines within the University of Manitoba. And so I would only be able to generalize my results to students taking sociology. Could I approach people sitting in some of the study areas or in the library or during a bison game? Well, no, because those would probably be all upper-level students. Those are the ones that tend to spend their time in the library or are mostly sports fans who also are students who are watching the bison game. Those all are not representative. So, the only way I can do it is in this case I would simply get the list of students and I would pick every fifth student and I would send a survey to every fifth student. That's entirely randomized. There's no conscious or unconscious bias present, and that would become representative if I would take those results and then generalize them to all University of Manitoba students, because I have really picked a random sample. Data Collection And here we're going to start talking about our methods of data collection. How do we gather the information? We're going to start with experimental methods. And what I'm going to do is I'm going to show you what is a classic experimental design. Now, what's important to know about experimental methods is that we manipulate the independent variable when we're doing an experiment in a way that we wouldn't manipulate the variables if we were doing surveys or field research, which we'll talk about in just a few minutes. So here we have a classic experimental design, and we're going to use the example of the effects of media violence on children, which is an area of specialization from my own study background. I worked on that in both my honors and my master’s degree. So, we have both an experimental group and a control group. So, we're going to have a group of kids, let's say, in this experiment, who are both male and female. And we'll have the experimental group and the control groups of two different groups of kids. We will do a pretest on both. So that pretest is going to be an interview with the child, an interview with the child's teacher, and an interview with the child's parents in which we are going to assess and determine the aggression level of that child before we take them to the experiment. So, we've done that. We do that for both the control group and the experimental group, and we have our aggression levels measured for each of the kids that's participating. And then for the children who are in the experimental group, they will now watch a violent video of some sort. The children who are in the control group will not watch the violent video, just the experimental group of children will. And then there will be a post-test both for the experimental group of children and the control group of children. The post-test. Each group will play a game of floor hockey and there will be observers present who will measure the aggression levels of the children who are playing floor hockey. And then that will be coded into an aggression level. Then we will take the pretest aggression results and the post stimulus test aggression results, and we will compare them to see if the children who are in the experimental group who did watch a violent video, if that affected their aggression levels. And you can see here the importance of the pretest. Some children might come in with very low aggression levels that spike significantly because of watching violent videos. Some may come in with very high aggression levels, and their aggression levels don't increase all that much or vice versa. The kids with low aggression levels, their aggression levels don't go up much at all after the violent video. Kids who already have high aggression levels, their aggression levels get higher. I don't know. That's the point of the experiment. And of course, to really make sure you are measuring what you think you're measuring, you have your control group in which we do the pretest and the post-test comparison because if those varied just as much then our results wouldn't mean as much because obviously means kids′ aggression levels just go up and down throughout the day. So that's why it's important to have the control group. If, as is the case in most of these studies, the control group, their pre and post aggression levels remain the same. But in the experimental group overall, their aggression levels go up. That does tell us something about cause and effect of watching violent videos. Now this is only this is not a long-term effect. This is only short-term effect. Right. We can't generalize that to you know, you play a violent game three days later, you do something violent. All we're measuring is immediate effects in this experiment. And it's important to make sure you qualify your results and be transparent about that. So that would be a classic experimental design. We also use lots of survey methods that's become synonymous with social science research. They may be questionnaires we mailed to someone, telephone interviews, in-person interviews, they may be focus groups, which is one of my favorites. I've used those quite often, and we have different types of questions that we would use. So, we would have the fixed, forced, or closed question where we might ask some basic information and we give you the categories that you can pick from. So, let's say I said, what province do you live in? Well, that doesn't allow someone who lives outside of our city, in all the provinces in Canada, that doesn't allow someone who lives outside of Canada to even answer that question, does it? Because I only want answers from people who live inside of Canada. So, it's a fixed, forced, or closed question. We may ask a rating question. We might say adults don't drive as well as teenagers, or teenagers don't drive as well as adults. And then we'll give you a couple of responses. Do you agree? Strongly agree. Do you disagree? Do you stand strongly disagree. So that's a rating. Where do you fit on that? Do you feel strongly about that or not? Very strongly at all. Might also use a ranking system. So, what is your favorite type of food? And we'll have, let's say, meats, pastas, bread, sweets. Those are the four categories I want you to pick your top one, your second favorite, your third and your least favorite. Clearly, I'm asking you to rank it again. This is also a bit of a fixed or forced question because you can't just write down whatever you want. I'm forcing you into categories. We also have semantic differential questions. How much do you enjoy the musical selection that was just played? So, I might have 0121210. And then you're going to circle. If you circle zero, it means you're right in the middle. You didn't enjoy and you didn't dislike. But if I've got an enjoy at this end, you might pick one. Or if you really didn't like it, you might pick two. That's a semantic differential. And then I'm just an open question. So, if we go back to the food to simply say, what's your favorite food? And you can put anything in there that you want to. Something that's critical is avoiding bias in the questions that we construct. So, we want to be careful about negative questions because negative questions often create sort of a double negative conundrum, making it a little bit unclear to the respondent, not what they know, what they want to answer. They just don't know which they should select. That represents their answer. So, if I say Canada should not support United Nations peacekeeping missions, do you agree or disagree? So, I disagree that Canada should not. That gets just a little bit confusing for the respondent. So, it's better to say Canada should support United Nations peacekeeping missions. And then there's a very clear. Yes, I agree with that. No, I don't agree with that option. It's not muddled by a double negative. Leading or loaded questions. Do you agree that the prime minister of Canada should increase family tax payments so that little children do not go to bed hungry? Okay. Sort of feels like there's only one answer to that. Even if you don't think child tax credits could be/should be increased, you don't feel like you want to disagree with that because there's also you want children to go to bed hungry part attached to the question. That's a really loaded question and we want to avoid doing that. It's a leading question. We want a certain answer from you. So, we're going to put that into the question to make you feel like that's really the only way you can answer it. Threatening questions should be avoided. Have you ever told a lie? Have you ever done any shoplifting? Those are those are threatening questions. Chances are, you're not going to get honest responses from nearly everyone. These questions might make the respondents uncomfortable or threatened, as it asks about potentially incriminating behavior. And then double-barreled questions, two different questions in one. Canada should spend less money on welfare programing and more money on education. I might agree with one part of that statement but disagree with another part of that statement because there's two things buried in that sentence. So, I don't really know how to answer. So that's a double-barreled question. More examples: Field research. Now, I want you to note the little yin yang image here on the bottom that my arrow is circling around because quantitative research and qualitative research are lagging and Yang they fit together well, even though they're very different in some ways. But qualitative research or field research is observing life in its natural habitat. You don't ask a bunch of questions of someone that are systematic. You don't manipulate anything the way you would in an experiment. So, field research is what we call qualitative and the other types of research I was giving you are typically quantitative, where there's measurements and there's statistics that you're generating, whereas in qualitative you're generating descriptions and interpretations by watching what's happening in a natural environment of some sort. And by natural I do not mean in the forest. I mean the example I'll use the street gangs, the natural habitat of a street gang. That's what you're observing. So, don't think forest or nature when you hear the word natural. Roles of the Observer What kind of role can an observer take? Well, an observer can be a complete participant, which means they're wholly concealed to the street gang members. They are a street gang member. As far as everybody else in that street gang understands it. They're just another member of the gang. So, they've joined this gang. Now, your hands are tied here in some ways because you can't have your notebook out and take notes on what you're observing, whatever time you spend with the street gang. Being a street gang member covertly without them knowing, you must take your notes later and hopefully remember everything. So that's one of the challenges of being a complete participant. There are some ethical challenges with this as well. Is it an invasion of their privacy to do it this way? Is there a problem with the kinds of deception you're using? Those are ethical questions. And what will be the impact on the participants when they find out that you were a researcher? What might be the effects on you when they find out that you were a researcher and sometimes a complete participant goes native, in other words, becomes a member of a street gang for real. They're no longer studying the group, they're now actually a member. There's also the participant as observer. So now you're with the street gang, but everybody knows your research street gangs. You'll still participate fully in all their events and all their activities. But everybody knows you're a researcher, so, there's a lot more transparency this way. You might be an observer only so; everybody knows you're a researcher. Just like with the participant as observer. But in this case, you're not participating with them. You're just off to the side in whatever context they're in, and you are observing what they're doing. You're not participating in what they're doing. And then finally, an undisclosed observer. So, you're not a member of the gang, which is a complete participant. You are watching them without them even knowing your present. Are you hiding somewhere or are you just observing them, you know, on a street corner, whenever you get a chance, in whatever way you can and they don't even know they're being observed, and they don't even know you're there. An undisclosed observer. Now to lock in on this regarding qualitative and quantitative research. These two things work beautifully together when it's possible you can generate all kinds of interesting data through experiments and through surveys. But another way to validate those results is to also do some kind of parallel field research. Both types are equally valuable in the world of the social sciences, though we probably have a leaning toward the quantitative. Anthropology does mostly qualitative, but over the last couple of decades, sociologists have increasingly leaned towards using more qualitative forms of research as well. So, it's good to understand both.