ENGE 3780 Applied Linguistics Course Outline PDF

Welcome! 1 WEEK 1 Introduction to research in applied linguistics Prof. Wilkinson ‘Wil’ Daniel Wong © University of Leicester Gonzales 4 September 2024 Prof. Wilkinson Gonzales 2 The plan… Introductions C...

Welcome! 1 WEEK 1 Introduction to research in applied linguistics Prof. Wilkinson ‘Wil’ Daniel Wong © University of Leicester Gonzales 4 September 2024 Prof. Wilkinson Gonzales 2 The plan… Introductions Course outline overview Lecture An overview of applied linguistics research methods 4 September 2024 Prof. Wilkinson Gonzales 3 Introduction Officially: Wilkinson Daniel Wong Gonzales But you can call me: Wil (preferred) Prof. Gonzales a “fourth-generation Chinese Filipino of Lannang heritage,” an ethnic minority in his home country of the Philippines 4 September 2024 Prof. Wilkinson Gonzales 4 My area of study “applied linguistics” = covers sociolinguistics according to some scholars 4 September 2024 Prof. Wilkinson Gonzales 5 Courses taught so far… 4 September 2024 6 Let’s go over the key things in the course outline… Check Blackboard for the latest version. Not just a regular outline! Contains links 4 September 2024 ENGE 5420 | Prof. Gonzales 7 Course Outline: ENGE 3780 A ‘living’ document. Will change from time to time. Link: https://tinyurl.com/ENGE3780- 2425 4 September 2024 Prof. Wilkinson Gonzales 8 What will I get out of this class? 1. Knowledge of concepts and procedures in applied linguistics 2. Ability to analyze small-scale qualitative and quantitative data to help you undertake independent research projects in applied linguistics 3. Ability to critically decide which method/approach to take or use Very broad, so we won’t go deep into a particular method… Learning by reading/presenting and doing! 4 September 2024 Prof. Wilkinson Gonzales 9 Important Links in the course outline (ENGE 3780) 4 September 2024 Prof. Wilkinson Gonzales 10 Class textbook(s) We have multiple! (Paltridge and Phakiti 2018; Dörnyei 2007; McKinley and Rose 2020) and articles for this course. Uploaded to the readings link for your convenience. Please do not share them with other students. 4 September 2024 Prof. Wilkinson Gonzales 11 Instructor and contact information 4 September 2024 Prof. Wilkinson Gonzales 12 Tips for communication w/ me You are welcome to attend any office hours held by me Face-to-face trumps email, but email is also okay. In general, you can expect me to answer a short email within 24 hours, except between 5 pm on Friday and 9 am on Monday or as otherwise noted. If my answer to your email will be more than a few sentences, I am very likely to ask you to come and chat face to face. 4 September 2024 Prof. Wilkinson Gonzales 13 Schedule: ENGE 3780 4 September 2024 Prof. Wilkinson Gonzales 14 Schedule: ENGE 3780 4 September 2024 Prof. Wilkinson Gonzales 15 Schedule: ENGE 3780 4 September 2024 Prof. Wilkinson Gonzales 16 Schedule: ENGE 3780 4 September 2024 Prof. Wilkinson Gonzales 17 Activities and Assessment: ENGE 3780 4 September 2024 Prof. Wilkinson Gonzales 18 Participation and engagement: ENGE 3780 Engagement in discussions led by the instructor and/or the facilitators. This can come in the form of questions or insights. You may participate orally in class and/or via tickets. (1,000 points) Engagement In-class workshops and practice problem sets (1,000 points) During tutorials, engage in contemporary readings in Applied Linguistics and contribute to insightful discussions. (2,000) Will be assessed holistically at the end of the semester based on level of engagement. 4000 points 4 September 2024 Prof. Wilkinson Gonzales 19 Participation tickets 4 September 2024 Prof. Wilkinson Gonzales 20 Graded problem sets In-class assessment Two parts: Quantitative (3000 points), Quantitative + Qualitative (3000 points) Semi-closed-book Characteristics Logic-oriented: Tests your thinking Practical Data-driven Minimal/easy computation Much will be derived from in-class problem sets, so don’t miss the in-class activities! 6,000 points 4 September 2024 Prof. Wilkinson Gonzales 21 Percentage to Final grade conversion If you attend classes and participate in activities diligently, you won’t get a C. I don’t like giving C grades. 4 September 2024 Prof. Wilkinson Gonzales 22 Some notes Check BlackBoard frequently if you did not enable announcements. Make sure to check your email frequently for announcements. If you have concerns about the class and want to get in touch with me privately, please contact me right after class. Too shy? Email me. Let’s get started! 4 September 2024 Prof. Wilkinson Gonzales 23 What is applied linguistics? What first comes to mind? Any thoughts? Recall your interview questions/preparations? 4 September 2024 Prof. Wilkinson Gonzales 24 Before we embark on this, let’s do an activity! Activity 1: What’s common? Case studies Use your skills of deduction to analyze the scenarios. Identify the underlying issue each scenario is trying to solve and how it relates to the study and use of language. By the end of this activity, you should be able to infer the broader goal that connects these scenarios, and how it relates to “Applied Linguistics.” 4 September 2024 Prof. Wilkinson Gonzales 25 Instructions 1. Skim through each scenario: Each scenario presents a situation that involves language in various contexts. You can work on this as a group or individually. It might help to highlight key ideas or patterns. 2. Connect to a Broader Goal: Reflect on how each scenario might be related to a larger goal or objective. Connect the dots. Answer the questions: 1. After analyzing all the scenarios, what common theme or goal did you observe? What are some patterns in these excerpts? What is the role of language? Reflect on how all these situations might be connected and what this implies about the study and application of language. 2. What do you think is the relevance of this to applied linguistics? 4 September 2024 Prof. Wilkinson Gonzales 26  Applied Linguistics (Grabe 2010) emerged as a language-centered problem-solving enterprise (Davies, 1999a) Focus: practical application of linguistics a discipline that… engages interdisciplinary resources (including linguistic resources) to address ________________ real life problems ESL - problem -> solution -> facilitate common goal social phenomena thr problemizing by oth factors -> collaboration of linguists 4 September 2024 Prof. Wilkinson Gonzales 27 The roots of applied linguistics Researcher-practitioners Educational psychologists who interested in exploring teaching were interested in exploring the and learning within language cognitive and psychological classrooms processes of language learning. 4 September 2024 Prof. Wilkinson Gonzales 29  language and sth. Applied Linguistics eg. lang and society lang and tech THEN NOW teaching and learning Interaction with disciplines such as technological business, politics, advancements and psychological sociology, globalization and cognitive anthropology, underpinnings of technology, medicine, and mobility, language multilingualism science learning (very broad!) applied linguistics is just beginning to consider its current scope and future directions Eg. Chatgpt 4 September 2024 Prof. Wilkinson Gonzales 30 The golden age of Applied Linguistics Over the last three decades: significantly matured in research. Now covers a broader range of topics, and research methods have also diversified. Growth driven by the global rise in mobility and multilingualism. Migration and globalization have led to increased demand for applied linguistics research to address language-related educational and social policy changes. 4 September 2024 Prof. Wilkinson Gonzales 31 Now, it is necessary to… take stock of what it means to ‘do research’ within applied linguistics theorize our available approaches, designs, methods, and data analysis techniques What we will be doing in this course! 4 September 2024 Prof. Wilkinson Gonzales 32 Combining research and Applied Linguistics.. What is the nature and scope of research IN applied linguistics? NATURE Interdisciplinary Linguistics, psychology, education, Text anthropology, sociology, political science, geography, philosophy Problem-solving at heart Mostly practical (but could stem from theory) 4 September 2024 Prof. Wilkinson Gonzales 33 Scope of Applied Linguistics Language education (teaching, learning, pedagogies etc.) Language policy and planning (schools, families, organizations etc.) Language in society (power relations, attitudes, identity, gender, politics) Second language acquisition (motivation, bilingualism etc.) Classroom research (feedback, teaching strategies etc.) Language testing and assessment Multilingualism (language ideologies, globalization, public spaces) Curriculum development 4 September 2024 Prof. Wilkinson Gonzales 34 What’s trending in Applied Linguistics? Research trends – most explored topics (2005- 2016) (Lei & Liu 2019) 4 September 2024 Prof. Wilkinson Gonzales 35 What’s trending in Applied Linguistics? Research trends – most explored topics (2005- 2016) (Lei & Liu 2019) 4 September 2024 Prof. Wilkinson Gonzales 36 What are the influential works in Applied Linguistics? Top 20 most cited publications (2005-2016) Lei & Liu 2019 4 September 2024 Prof. Wilkinson Gonzales 37 What are the influential works in Applied Linguistics? Top 20 most cited publications (2005-2016) Lei & Liu 2019 4 September 2024 Prof. Wilkinson Gonzales 38 Who are the influential people in Applied Linguistics? Top 20 most cited scholars (2005- 2016) Lei & Liu 2019 4 September 2024 Prof. Wilkinson Gonzales 39  Some big names in the field… Rod Ellis Merrill Swain second language acquisition, Jim Cummins Zoltán Dörnyei Second language acquisition, Language, power, SLA and language teaching and teacher Output hypothesis, pedagogy, identity, psychology, education communicative competence bilingualism motivation in second language learning Ken Hyland Michael Long Nick Ellis academic discourse, second Douglas Biber Interaction Hypothesis, SLA corpus linguistics, Stylistic variation, Corpus language writing, English for SLA, psycholinguistics, psycholinguistics, linguistics, English typology, Academic Purposes focus on form computational modeling discourse, English grammar 4 September 2024 and cognitive linguistics Paradigms and Applied Linguistics Research What is a paradigm? “the philosophy supporting the knowledge or reality a researcher uses to understand a phenomenon” (McKinley, 2020, p. 2) a set of assumptions or values about the nature of reality, knowledge, and research methods 4 September 2024 Prof. Wilkinson Gonzales 41 A concrete example… What is this? … a tool for murder? … a bargaining chip? Different assumptions, … a tool for nailing things? Different values … a museum artifact? 4 September 2024 Prof. Wilkinson Gonzales 42 Some misconceptions Research is NOT done in vacuum. Research is NOT just about the analysis of data. It is a stance-taking project. 4 September 2024 Prof. Wilkinson Gonzales 43 Paradigm: Who you are as a researcher… What is reality nature of reality ONTOLOGY Researchers' views on the nature of reality What do we think we can know? What is reality? you and the object object EPISTEMOLOGY METHODOLOGY Researchers’ views on the nature of knowledge and how it Researchers’ approach to can be acquired investigate the reality What is our relationship to the how do we go about our pursuit of thing we are trying to know? knowledge? can you measure? can you touch? can you claim ownership? defien ontology adn epistemolgy w question 4 September 2024 Prof. Wilkinson Gonzales 44 Put simply… ONTOLOGY: What we’re studying What is reality? Nature of reality How do you see the social reality that you wish to investigate? “composition of the universe” EPISTEMOLOGY: Approach for understanding How do we know about reality? understanding the methods, validity, and scope of gaining knowledge about that reality. “understanding and knowledge of that universe” actual measuring METHODOLOGY: Actual steps How do we go about finding out? specific methods, strategies, and tools used in research to investigate and gather data about reality practical application of epistemology 4 September 2024 Prof. Wilkinson Gonzales 45 Research paradigms (Troudi 2021; Dong and Dong 2024) Ontology Epistemology Methodology Reality is… Knowledge or Reality can be investigated evidence is… through… Positivism objective and observable and empirical, quantitative quantifiable measurable methods, such as phenomena experiments and black and white surveys Post- conjectural, probabilistic, mixed methods, but positivism only and hypotheses primarily quantitative approximated can be falsified with experimental uncertainty, fill in gaps of existing control. possibility Critical shaped by influenced by qualitative methods, inquiry social, political, human critical discourse and cultural interests and analysis, and values power relations ethnography 4 September 2024 Prof. Wilkinson Gonzales 46 Research paradigms (Troudi 2021; Dong and Dong 2024) Ontology Epistemology Methodology Reality is… Knowledge or Reality can be investigated individual is at the centre, there is no evidence is… through… inherent reality. Reality is a social construction by every human beings. Interpretivism/ subjective, subjective, qualitative methods, constructivism constructed by created through such as case studies, individuals interaction and interviews, and interpretation participant observation Pragmatism practical and based on mixed methods tailored based on actions, to specific practical actions, outcomes, and outcome situations, and practical impacts consequences Participatory co-constructed collaboratively participatory methods, by researchers developed and action research, and participants shared collaborative inquiry immersion say oxford, we develop materials with oxford individuals then it is participatory 4 September 2024 Prof. Wilkinson Gonzales 47 Activity: Case analysis in graded problem set Objective: To help students learn how to identify different research paradigms and understand how these paradigms are connected to ontology (the nature of reality), epistemology (the nature of knowledge), and methodology (the approach to research). 4 September 2024 Prof. Wilkinson Gonzales 48 Instructions Group Work: Students will be divided into groups. Each group will work on two cases from the six provided. These cases will be distributed evenly. Analyze Each Case: For each case, identify the research paradigm being applied. Then, explain how the paradigm is related to the researcher's views on ontology, epistemology, and methodology. These are the paradigms that have been discussed in class. Positivism Post-positivism Critical inquiry Interpretivism/constructivism Pragmatism Participatory Class Discussion: After doing this, each group will share their analysis with the class, and we will discuss how different paradigms shape research approaches. 4 September 2024 Prof. Wilkinson Gonzales 49 Table to be filled out… 4 September 2024 Prof. Wilkinson Gonzales 50 Questions to ponder on… Why does research in Applied Linguistics need to be informed by philosophical perspectives? Do you think researchers in Applied Linguistics will benefit from the discourses of research paradigms? How? Can one be informed by more than one paradigm at the same time? Do researchers state what paradigms inform their studies when publishing their research? Why or why not? 4 September 2024 Prof. Wilkinson Gonzales 55 What to expect for the following weeks… (Scientific) research methods in Applied Linguistics The anatomy of research and how to develop it Collection: Survey/questionnaire method QUANTITATIVE Collection: Experimental method Collection: Corpus method Analysis: Basic statistics Grounded Theory Method (GTM) Collection: Interview/focus group QUALITATIVE Analysis: Thematic/content analysis Collection: Ethnography/observation Analysis: Case study approach Analysis: Critical Discourse Analysis (CDA) MIXED Mixed methods research and analysis 4 September 2024 Prof. Wilkinson Gonzales 56 The plan… Course logistics Recap Research dissected! 1 WEEK 2 The anatomy of research and how to develop it Prof. Wilkinson ‘Wil’ Daniel Wong Gonzales © University of Leicester 12 September 2024 Prof. Wil Gonzales 2 Course Logistics Tutorials next week Bring Problem Set 2 (we will complete PS2c) Class notes/slides Uploaded to Blackboard OneDrive three to five hours before class (or earlier) Recordings All class lectures will be recorded via Panopto (check Blackboard> Panopto Video) Tickets No need to write one every week … especially if you are participating actively in class If you do write one, please submit to Echo after class. 12 September 2024 Prof. Wil Gonzales 3 Recordings via Panopto 12 September 2024 Prof. Wil Gonzales 4 This week: Research dissected Beginning research with the research “conceptualization” Common stages/steps Common elements 12 September 2024 Prof. Wil Gonzales 5 How do to research: A guide Topic Literature Research Research selection review problem purpose Ethical Context and Research Research considerations methods methodology questions Data Interpretation Research collection/field Data analysis /conclusion communication work Not a linear process, but cyclic and reiterative 12 September 2024 Prof. Wil Gonzales 6 Overall guidelines for Applied Linguistics research 12 September 2024 Prof. Wil Gonzales 7 Guidelines about the research topic 1. The topic must have clearly stated aims and questions : PURPOSE 2. The topic must relate to existing knowledge and needs : RELEVANCE 3. The topic must fit the resources available : FEASIBILITY 12 September 2024 Prof. Wil Gonzales 8 How should we select our approach? The approach must protect the interests of participants: ETHICS 12 September 2024 Prof. Wil Gonzales 9 How about research design? The design must be aware of its underlying assumptions: PHILOSOPHY Ontology, epistemology, positivism, interpretivism, critical realism, pragmatism, research paradigms 12 September 2024 Prof. Wil Gonzales 10 Good research methods have the following qualities… The methods need to be collected and used in a justifiable way: ACCOUNTABILITY 12 September 2024 Prof. Wil Gonzales 11 When conducting research, your findings should ideally: 1. Contribute something new to knowledge: ORIGINALITY 2. Be cautious about their claims: PROOF 12 September 2024 Prof. Wil Gonzales 12 Anatomy of a research paper 1. 2. Title—concise, clear and specific Authors and institutions IMRD 3. Abstract: purpose, method, major findings and implications 4. Introduction better to have a separate section—thematic presentation is better) 5. Methodology—describe research design, methods, sample, tools, settings, timeline, data collection and data analysis procedures etc. 6. Results: Findings from the study—be selective and include the most striking data— thematize the findings 7. Discussions and conclusions: This section is not just a summary—it show discuss the theoretical meaning of the findings from the study You can be as reflective as possible to discuss what lessons have you learned regarding the topic you have investigated and how this study contributes to the field of your research. What were the limitations? What theoretical, practical and/or policy-related recommendations would you like offer? 12 September 2024 Prof. Wil Gonzales 13 Topic and title TOPIC TITLE general area of research specific-–the name of your research Must be concise, specific, focused, clear variables (quantitative) and constructs (qualitative) 12 September 2024 Prof. Wil Gonzales 14 Which are more desirable titles and topics? “Languages Matter: An Investigation into the Significance of Languages” “Identity construction and multilingual self of Chinese- English bilingual learners” “English vs. French: A Comparative Study of Vocabulary” “The effectiveness of corrective feedback in academic writing at CUHK” “Tongues of Belonging: Exploring the Interplay between Mandarin Use and Chinese-American Identity in New York Chinatown” 12 September 2024 Prof. Wil Gonzales 15 The million-dollar questions How do I come up with a topic? Where do new ideas come from? 12 September 2024 Prof. Wil Gonzales 17 No definite answer, but here are some possible sources… Research ideas that originate from within ourselves personal experiences of language-related issues—learning, use, discourse, politics, community, public place Research ideas that originate outside ourselves course instructors/professors joint projects with professors/friends conferences/talks/discussions Research ideas that originate from research in the field Reading recent books/articles/research reports 12 September 2024 Prof. Wil Gonzales 18 More questions to guide your topic selection… What’s new and interesting? Is it researchable in terms of time resources and context? Can you collect data? Are you really interested in it? 12 September 2024 Prof. Wil Gonzales 19 Activity: Concept mapping (Problem Set 2a) 12 September 2024 Prof. Wil Gonzales 20 Selecting and narrowing down the topic General topic/area: language anxiety First level narrowing: What in language anxiety? Speaking anxiety? Writing anxiety? Second level narrowing: What’s the specific issue/problem? Who has and where is the problem? 12 September 2024 Prof. Wil Gonzales 21 Research problem “a problem that someone would like to research” (Fraenkel et al., 2012). theoretical practical methodological “the issue that exists in the literature, in the theory, or in practice that leads to a need for the study” (Roberts, 2004) 12 September 2024 Prof. Wil Gonzales 22 Statement of research problem 1. Synthesize the literature review 2. Identify the problem in the existing literature (void or gap) 3. Explain why it is a problem 4. Explain why it is important to study the problem 12 September 2024 Prof. Wil Gonzales 23 The research problem in relation to the study General area under study Gap/Problem What we know and don’t know about the problem Your specific study (here, you find your pathway) Purpose Research questions 12 September 2024 Prof. Wil Gonzales 24 Example: Improved student performance K-university continues to be a national priority The availability of qualified teachers is critical to solving this problem. Past students have shown a number of reasons for the teacher shortage: pay working conditions, support, recognition and reward There is also a shortage of qualified and available substitute teachers Past students indicate various reasons for the shortage of substitutes: pay, working conditions, support, recognition, reward, and student behavior. What isn’t known is whether the decisions of current substitutes to stay in or leave teaching are the same as those in past studies The purpose of this study is ….. (Gonzales, 2002) 12 September 2024 Prof. Wil Gonzales 25 Activity (Problem Set 2b) Mastering the Art of Crafting Effective Problem Statements Instructions Reading and Analysis: 1. Read the two provided problem statements from actual articles. 2. Highlight the key parts that make each problem statement effective. 3. Justify your highlights 4. Enumerate: what are the qualities of a good research problem statement? 12 September 2024 Prof. Wil Gonzales 26 Revision 1. Review the poorly written problem statement provided. 2. Revise it to make it clearer and more effective, using what you've learned from the previous examples. 12 September 2024 LGS | Prof. Gonzales 28 How does it fit in my research proposal? The statement of the research problem, together with the precise exposition of the research questions and/or hypotheses = a transition between the review of literature and the description of the methods of the study. 12 September 2024 Prof. Wil Gonzales 29 Guide: Research questions and/or hypotheses 1. State and justify overarching and specific questions 2. Be concise. 3. Should be relevant to the problem. 4. Answerable – qualitative and/or quantitative 12 September 2024 Prof. Wil Gonzales 30 The link between research questions and the topic When you “Think Like a Researcher” you are focused on a research question rather than a research topic. You must also be able to articulate the problem that underlies the question. The solution you argue for in your proposal must be arguable and feasible based on reliable evidence. 12 September 2024 Prof. Wil Gonzales 31 Discuss: Are these good questions? 1. What are the factors that influence the choice of language in the family? 2. Why do parents send their children in EMI schools? 3. How do teachers deal with students’ learning needs in after- school programs during an average week? 4. Why do local people give a high social value to English? 5. Does corrective feedback help EFL learners improve their speaking proficiency? 12 September 2024 Prof. Wil Gonzales 32 A working criteria for good research questions # Your question is… Meaning… is not easily answered with a It has some substance and requires 1 simple yes or no. explanation. has an underlying problem with It is important to someone other 2 social significance (local, than just you! national or international). poses a genuine question and It avoids using loaded language or 3 aims for neutrality. suggesting a pre-determined answer. It is re-searchable. Others have can be answered with reliable 4 already been contributing to this evidence. conversation. It is not too narrow, nor too broad; it 5 has appropriate scope. does not leave you with too much or too little information. If you have an existing research question, see if your question is TRUE for these five criteria. If so, then your research question is probably workable. 12 September 2024 Prof. Wil Gonzales 33 Research questions and/or hypotheses Could be explicitly listed as questions 12 September 2024 Prof. Wil Gonzales 34 Research questions and/or hypotheses … or could be incorporated in the text as a declarative sentence. 12 September 2024 Prof. Wil Gonzales 35 Activity: Piecing it all together – Tutorials! (Problem Set 2c) 12 September 2024 Prof. Wil Gonzales 36 Research hypotheses A prediction, a statement of what specific results or outcomes are expected to occur. Linked to your research questions! Should clearly indicate any relationships expected between the variables being investigated 12 September 2024 Prof. Wil Gonzales 37 Research hypotheses in quantitative research Null hypothesis There is no statistically Alternative hypothesis significant relationship There is a significant between the tested relationship between the variables tested variables. Example: Effect of essay type on Example: Effect of gender on use number of nouns of A variant (-ise) vs. B variant (-ize) 100% 50% 0% Essay 1 Essay 2 Nouns Total words 12 September 2024 Prof. Wil Gonzales 38 So far, we have talked about… Guidelines for social research Anatomy of research Research topic and title Problem statement/research problem Research questions Research hypotheses 12 September 2024 Prof. Wil Gonzales 39 From proposal to article 1. 2. Title—concise, clear and specific Authors and institutions IMRD 3. Abstract: purpose, method, major findings and implications 4. Introduction better to have a separate section—thematic presentation is better) 5. Methodology—describe research design, methods, sample, tools, settings, timeline, data collection and data analysis procedures etc. 6. Results: Findings from the study—be selective and include the most striking data— thematize the findings 7. Discussions and conclusions: This section is not just a summary—it show discuss the theoretical meaning of the findings from the study You can be as reflective as possible to discuss what lessons have you learned regarding the topic you have investigated and how this study contributes to the field of your research. What were the limitations? What theoretical, practical and/or policy-related recommendations would you like offer? 12 September 2024 Prof. Wil Gonzales 40 The introduction (which some argue is the hardest) Introduces the topic Provides the general scholarly context of the topic. Here you identify and state the GAP in the existing literature: research problem (see next slide, though) 12 September 2024 Prof. Wil Gonzales 41 Let’s be more specific: outline of the introduction Step 1: State why this problem is interesting to your field It is generally acknowledged that textbooks play an important role in language classes. For example, Author (19x) states that textbooks provide significant language input. 12 September 2024 Prof. Wil Gonzales 42 Let’s be more specific: outline of the introduction Step 2: State the problem/contribution/gap or question using words such as: however, nevertheless, yet, but Nevertheless, some researchers (Author, 19xx; Author, 20xx) have shown that many text authors employ artificial dialogues that lack normal language features. 12 September 2024 Prof. Wil Gonzales 43 Let’s be more specific: outline of the introduction Step 3: State the purpose of your paper The purpose of this paper is to investigate current textbooks comparing their dialogues with those of native speakers. 12 September 2024 Prof. Wil Gonzales 44 Let’s be more specific: outline of the introduction Step 4: State the theory, research design and context I draw on the theory of ‘authentic language’ (Author, 19xx) to analyze how students in [context] compare and judge the relevance of language used in everyday language and textbook dialogue. I adopt a case study of three non- native speakers from [context]. 12 September 2024 Prof. Wil Gonzales 45 Wil’s tips for good introduction writing Reconsider Gap-filling: Go beyond ‘there is little research in X area’ Focus on the merits and contributions of the paper to theory and understanding What is so special about your study? Do the introduction last! Easier to write papers knowing what you have already found. Ultimate goal: Clarity and engagement (Feel free to use “I” pronouns and short prose) 12 September 2024 Prof. Wil Gonzales 46 The literature review Identify the most relevant studies (journals/books/research reports/newspapers/conference papers) Provide a brief and analytical review of studies (purpose, theory, method, major findings…) Demonstrate the linkage between the previous studies and the proposed study. Describe the relevance of the literature for the proposed study. In what ways, the previous studies inform your understanding about the proposed study? Identify the gap or void—purpose, theory, method, context 12 September 2024 Prof. Wil Gonzales 47 Everything is linked! STRUCTURES Research DEVELOP STRUCTURES question Literature Hypotheses Findings review USED TO Purpose STRUCTURE USED TO DISCUSS INFORMS/JUSTIFIES 12 September 2024 Prof. Wil Gonzales 48 Next few weeks Collection: Survey/questionnaire method Collection: Experimental method Collection: Corpus method Analysis: Basic statistics 12 September 2024 Prof. Wil Gonzales 49 The plan… Course logistics Lecture Problem set 3c 1 Survey/questionnaire method Prof. Wilkinson ‘Wil’ Daniel Wong Gonzales 18 September 2024 Prof. Wil Gonzales 2 Quick recap What is ‘research design’? “An operating model or blueprint for a research project, which accounts for internal reasoning (causality) and external reasoning (generalizability)” (Griffee, 2012) Contains directions to determine the answer. 18 September 2024 Prof. Wil Gonzales 3 Multiple research designs… Survey Experimental Ethnography Grounded Theory Corpus-based Narrative Inquiry Discourse Case Study Critical approach Analysis 18 September 2024 Prof. Wil Gonzales 4 Quick recap: How do I choose a design? Two considerations Practical Political 18 September 2024 Prof. Wil Gonzales 5 Quick recap: How do I choose a design? Practical Political Depends on your Select the design research purpose you already know and questions. from colleagues etc. Which design best Design promoted by answers your senior researchers? questions? You could also be intrigued by certain What is your ontology and design and want to epistemology? learn how it works 18 September 2024 Prof. Wil Gonzales 6 What is survey design? A methodology used to systematically collect data from a sample of participants to gather information and insights about various topics or phenomena. involve the use of structured questionnaires or interviews that consist of a set of predetermined questions often with standardized response options. 18 September 2024 Prof. Wil Gonzales 7 Structured survey questionnaires What do you think is the research question here? 18 September 2024 Prof. Wil Gonzales 8 Interviews (face-to-face surveys) “How would you improve Hong Kong?” 18 September 2024 Prof. Wil Gonzales 9 What is the goal? Gather data that can be analyzed statistically to answer research questions and draw conclusions. 18 September 2024 Prof. Wil Gonzales 10 Interviews > Coded data ”How do we improve HK?” Improve Econom Give roads y more money Person 1 1 0 0 Person 2 0 1 0 Person 3 0 0 1 18 September 2024 Prof. Wil Gonzales 11 Survey design: Key features Measures specific constructs that may be theoretical or practical Theoretical: opinion, beliefs, attitudes Objective/practical: ownership of certain objects, time spent on certain tasks useful for testing hypotheses and identifying patterns. 18 September 2024 Prof. Wil Gonzales 12 Other key features Good for collecting quantitative data efficiently, easily, and systematically suitable for research with resource and time constraints Can be used to target a larger sample size to achieve generalizability 18 September 2024 Prof. Wil Gonzales 13 Other key features Standardized or streamlined questions/items: ensure consistency minimizes bias and allows for straightforward data analysis 18 September 2024 Prof. Wil Gonzales 14 Some key components Sampling Pilot testing Statistical Visual reporting analysis 18 September 2024 Prof. Wil Gonzales 15 Sampling Researchers select a subset of the population (sample) to participate in the survey (ideally randomized) More on this later… 18 September 2024 Prof. Wil Gonzales 16 Some key components of survey design Sampling Pilot testing Statistical Visual reporting analysis 18 September 2024 Prof. Wil Gonzales 17  Summarized and reported, often in the form of tables, charts, and statistical summaries. Column/bar chart Box plots Text tables Comparison charts Box plots 18 September 2024 Prof. Wil Gonzales 18 Common visual representation types in AEL Text Table: Best for presenting exact values, especially for small datasets or when comparing individual items. Pie Chart: Ideal for showing the proportion of categories in a whole. Best when there are few categories. Column/Bar Chart: Useful for comparing quantities across categories. Box Plot: Useful for showing the distribution of data and identifying outliers. 18 September 2024 Prof. Wil Gonzales 19 Common visual representation types in AEL Line Chart: Best for displaying data trends over time. Comparison Chart: Useful for showing differences and similarities between groups or items. Histogram: Used to show the distribution of a dataset, often for continuous data. Scatter Chart: Great for showing relationships between two continuous variables. 18 September 2024 Prof. Wil Gonzales 20 Does visual representation matter? 18 September 2024 Prof. Wil Gonzales 21 Yes!!! 18 September 2024 Prof. Wil Gonzales 22 Visualizing survey design General steps Construct 1. Planning the survey, 2. Developing the instrument Population (includes piloting to check of interest validity and reliability) 3. Gathering the data, Sample 4. Analyzing the data statistically, 5. Analyzing the data qualitatively, Instrument 6. Reporting the results. Brown (2001, p. 8) 18 September 2024 Prof. Wil Gonzales 23 Planning the survey Identifying Identifying and defining research problem and Teachers are quitting their progression early. defining Identifying research questions RQ What are the factors that are forcing teachers to quit their profession? Selecting the sample and sampling procedure 500 Selecting Population and sample teachers 50 teachers 18 September 2024 Prof. Wil Gonzales 24 Sampling Researchers select a subset of the population (sample) to participate in the survey (ideally randomized) should be representative of the larger population to ensure generalizability of findings. 18 September 2024 Prof. Wil Gonzales 25 Probability sampling “every member of the population has a chance of being selected” mainly used in quantitative research. If you want to produce results that are representative of the whole population, probability sampling techniques are the most valid choice. 18 September 2024 Prof. Wil Gonzales 26 Sampling methods: Probability sampling every member of the individuals are chosen population has an at regular intervals. equal chance of being selected. Your sampling frame should include the whole population dividing the population Instead of sampling into subpopulations that individuals from each may differ in important subgroup, you randomly ways. select entire subgroups. draw more precise conclusions by ensuring that every subgroup is properly represented in the sample. 18 September 2024 Prof. Wil Gonzales 27 Issues with probability sampling? limited resources privacy concerns probability sampling methods might require disclosing more information about potential participants heterogeneous populations assume that all members of the population have an equal chance of being selected. If the population is highly diverse or heterogeneous, this assumption might not hold true 18 September 2024 Prof. Wil Gonzales 28 Issues with probability sampling? specialized subgroups rare or hard-to-reach subgroups within the population exploratory research non-probability sampling methods might be used to quickly gather preliminary data nonresponse issues can sometimes result in nonresponse bias if a significant portion of selected participants do not respond 18 September 2024 Prof. Wil Gonzales 29 Non-probability sampling individuals are selected based on non-random criteria, and not every individual has a chance of being included. Easier and cheaper to access, but it has a higher risk of sampling bias. inferences you can make about the population are weaker than with probability samples 18 September 2024 Prof. Wil Gonzales 30 Non-probability sampling GOAL: not to test a hypothesis about a broad population, but to develop an initial understanding of a small or under- researched population. 18 September 2024 Prof. Wil Gonzales 31  Sampling methods (Non-probability) researcher using includes the their expertise to individuals who select a sample that happen to be most is most useful to accessible to the the purposes of the researcher. research. divide the population recruit into mutually participants via exclusive subgroups other participants and then recruit sample units until you reach your quota 18 September 2024 Prof. Wil Gonzales 32 Sample size? No clear-cut answer Larger the better? 1-10% of the total population Depends on how scientific is the sampling procedure 18 September 2024 Prof. Wil Gonzales 33 Arbitrary criteria… 100 for a descriptive study, 50 for a correlational study, and 30 in each group for experimental and causal-comparative studies. Dorney (2007): 30 for correlation research, 15 for each group in experimental research, and 100 for descriptive multivariate procedure 18 September 2024 Prof. Wil Gonzales 34 Tools development QUESTIONNAIRE OBSERVATION INTERVIEW ACTIVITY SHEETS CHECKLIST PROTOCOL (STRUCTURED) 18 September 2024 Prof. Wil Gonzales 35  Types of survey data Descriptive data: demographic information (e.g. respondent’s age, gender, place of birth, level of education, and ethnicity) Behavioral data: actions that respondents do–frequency, choice, Preference: opinion, perspectives, attitudes etc. 18 September 2024 Prof. Wil Gonzales 36 Types of multiple choice questions… ONE OPTION How motivated are you in your work as a teacher? 1. Not motivated at all 2. Slightly motivated 3. Moderately motivated 4. Highly motivated MULTIPLE OPTIONS If you want to quit teaching, what could be the possible reasons? Check all that apply. Low salary Lack of institutional support as expected Low social prestige of teaching job Political interference in the school Teachers are not involved in making decision Family obligations Feeling saturated 18 September 2024 Prof. Wil Gonzales 37 Advantages 18 September 2024 Prof. Wil Gonzales 38 Ranking questions How do you rank the following statements in term of their influence on your motivation for teaching? Items Lowest = 1 Moderate = 2 Highest = 3 a. School working environment/culture b. Feedback and support mechanism c. Salary and benefits d. Opportunities for professional development e. Family support f. Social status and recognition g. Association with professional organizations 18 September 2024 Prof. Wil Gonzales 39 Rating scale/Likert scale a rating scale used to measure opinions, attitudes, or behaviors. 18 September 2024 Prof. Wil Gonzales 40 Likert scale 18 September 2024 Prof. Wil Gonzales 41 18 September 2024 Prof. Wil Gonzales 42 18 September 2024 Prof. Wil Gonzales 43 Activity (Problem Set 3c) TEACUP Olympics Designing field questions = critical part of the research process, requiring both precision and skill Mastering this skill doesn't have to be difficult. The Teacup Olympics is designed to guide you through this essential step in your project, helping you identify and refine mediocre questions—even those you've written yourself. 18 September 2024 Prof. Wil Gonzales 44 18 September 2024 Prof. Wil Gonzales 45 Task: Group yourselves, and then write a survey question based on the information above. If your group has five people, then, there must be five questions. Write these questions on a piece of paper, and submit it to the professor/instructor after. 18 September 2024 Prof. Wil Gonzales 46 Introducing T.E.A.C.U.P. Tolerable not too threatening Essential for testing one or more of the hypotheses Answerable within the knowledge and thought pattern of the respondent Clear to the respondent not just the researcher Unbiased leaving all options open for the respondent Penetrating revealing what the researcher really wants to discover 18 September 2024 Prof. Wil Gonzales 47 Tolerable: not too threatening Bad example. About how often do you exaggerate the amounts on your expense reports: (a) usually or always (b) sometimes (c) Rarely (d) seldom or never. Good adjustment or alternative. How often do you think people in this company exaggerate the amounts on their expense reports 18 September 2024 Prof. Wil Gonzales 48 Essential: For testing one or two hypotheses Hypothetical central research question: “What does the staff of the Nigerian Revenue Authority believe are the appropriate incentives that can motivate the building of a culture of integrity in the department?” Hypothesis An open system of promotions would be a strong incentive. 18 September 2024 Prof. Wil Gonzales 49 Essential: For testing one or two hypotheses Bad example. Agree or disagree: The integrity of the department is worse than it was 5 years ago because the country's economy is worse in general. 1 2 3 4 5 (1-disagree, 5-agree) Good adjustment or alternative. A more open system of staff promotions would be a strong incentive for staff to work with integrity. 18 September 2024 Prof. Wil Gonzales 50 Answerable: Within the Knowledge and Thought Pattern of the Respondent Bad example. (posed to a fourth grader). “What are the most useful improvements your teacher has made during this school year?” Good adjustment or alternative. “What does your teacher do to make boring subjects more interesting for the class?” 18 September 2024 Prof. Wil Gonzales 51 Clear: to the Respondent Not Just the Researcher Bad example. How much would our effectiveness go up if we had more training? (a) a lot, (b) somewhat, (c) a little. Good adjustment or alternative. How much would our effectiveness go up if we had more on-the-job training provided during our regular working hours? (a) a lot, (b) somewhat, (c) a little. 18 September 2024 Prof. Wil Gonzales 52 Unbiased: Leaving All Options Open for the Respondent Bad example. Is your marriage difficult these days, or is it worse than that? Good adjustment or alternative. On a scale of 1 to 10 with 10 being most positive, how are you and your wife getting along? (Note that in some circumstances, this question might still violate the “Tolerable” criterion.) 18 September 2024 Prof. Wil Gonzales 53 Penetrating: Revealing What the Researcher Really Wants to Discover Bad example. Why didn't the project work out as planned? Good adjustment or alternative. Would you say that the project's failure was mostly due to inadequate staffing? Why or why not? 18 September 2024 Prof. Wil Gonzales 54 Mechanics To start the game, the instructor will randomly select the first contestant and a panel of six judges, each representing one of the TEACUP criteria. The contestant has the option to identify themselves. The contestant or instructor will then read the survey question to the panel. Each judge will provide a score based on their assigned TEACUP criterion. Scoring: 4 = excellent, 3 = good, 2 = acceptable, 1 = poor, 0 = disqualified 18 September 2024 Prof. Wil Gonzales 55 Mechanics Judges score the question focusing on their assigned criteria. Spectators also score: Spectators rate the question using all six TEACUP criteria. Spectators use the provided scorecard to practice applying all criteria simultaneously. 18 September 2024 Prof. Wil Gonzales 56 Mechanics the instructor ask judges display their scores using their fingers, keeping them visible during the discussion. Judges comment first, followed by spectators, who may ask for explanations or suggest alternative scores. The contestant then has a chance to speak. Contestants are discouraged from justifying their questions with phrases like "What I meant was..." to encourage a focus on rephrasing questions to be clearer and more effective. And the game continues… 18 September 2024 Prof. Wil Gonzales 57 Administration of survey Online tools Qualtrics Survey Monkey Google Forms SoGoSurvey Typeform Zoho Survey Survey Gizmo Survey Planet Qualtrics Telephone survey Face-to-face survey Mail survey 18 September 2024 Prof. Wil Gonzales 58 Some areas where surveys are used Needs analysis Program evaluation Language test Perceptions, beliefs, opinions and attitudes Motivation and anxiety Learning strategies Sociolinguistic situation Other areas? 18 September 2024 Prof. Wil Gonzales 59 Reflection: Are surveys always the best tool to use? Discuss: If they’re efficient and quick, then aren’t all research using survey methods? What undesirable actions do participants do when doing surveys (from you)? 18 September 2024 Prof. Wil Gonzales 60 Some downsides of survey methods… Limited Depth restrict participants' ability to provide in-depth or nuanced responses for complex or multifaceted topics Response Bias Participants may provide inaccurate or socially desirable responses Lack of Context often lack the rich contextual information provided by qualitative methods like interviews or ethnography Rigid the questions cannot be changed when new insights emerge. 18 September 2024 Prof. Wil Gonzales 61 The plan… Course logistics Lecture and problem sets! 1 Logistics Optional activity: October 16, 2024 (supposed to be no classes) Applied Linguistics in the Metaverse (LSK 304-306) 6:45 to 9:15 PM Need to get a head count (500 points extra if you want to participate) 26 September 2024 2 Let’s begin with this… PS 4b-1 Research shows that language learners with high self- confidence often achieve greater fluency than those with low self-confidence. Does this relationship imply that higher self-confidence leads to better fluency in the target language? Or does it suggest that better fluency leads to higher self-confidence? Explain your reasoning, and identify the general issue. 26 September 2024 Prof. Wil Gonzales 3 Often in survey methods we find similar patterns…. You may find that students who self-rated themselves with HIGH self-confidence are also students who self- rated HIGH fluency in the target language. Can’t really tell if one causes the other…. 26 September 2024 Prof. Wil Gonzales 5 Downsides of survey method… Downside Description Causality Surveys often struggle to establish causality. They do not allow for manipulation of variables to observe changes in outcomes. Response bias Respondents may answer dishonestly or inaccurately. Lack of control Surveys lack the ability to control extraneous variables. Social Respondents may provide socially acceptable desirability bias answers instead of truthful ones. Replication Surveys can be difficult to replicate under identical conditions. These are things that another method can help with!!! 26 September 2024 Prof. Wil Gonzales 6  EXPERIMENTS! 26 September 2024 Prof. Wil Gonzales 7 WEEK 4 Experimental method Prof. Wilkinson ‘Wil’ Daniel Wong Gonzales 26 September 2024 Prof. Wil Gonzales 8 What is an experiment? method designed to investigate causal relationships between variables manipulating one or more independent variables to observe their effects on one or more dependent variables while controlling for other factors. 26 September 2024 Prof. Wil Gonzales 9 What makes an experiment an experiment? Problem Set 4a Dr. Smith conducted a study examining the relationship between the amount of exposure to English-language media and the fluency in spoken English of young learners. A sample of thirty 4-year-old children from a local preschool was used in the study. English media exposure was measured by interviewing the parents about each child’s media consumption habits. Based on the results of the interview, each child was then placed into one of two groups: high exposure to English media and low exposure to English media. Fluency in spoken English was measured by observing the children during a regular preschool afternoon. Finally, Dr. Smith compared the fluency levels for the high- exposure group with the fluency levels for the low-exposure group. Explain why Dr. Smith’s study is not an example of the experimental research strategy. 26 September 2024 Prof. Wil Gonzales 10 Could this be an experiment? Why or why not? 26 September 2024 Prof. Wil Gonzales 13 Experiment components: the basics 26 September 2024 Prof. Wil Gonzales 14 Experimental Designs in Applied Linguistics 26 September 2024 Prof. Wil Gonzales 15 From start to finish… 26 September 2024 Prof. Wil Gonzales 16 Hands-on with experiments! (PS 4c) This activity introduces you to the concept of experiments by using paper airplanes. You will learn how to manipulate one factor (the independent variable) to observe how it affects an outcome (the dependent variable). Additionally, you will see how these experimental concepts apply to research in Applied Linguistics, such as testing different teaching methods to see which is most effective. Please refer to Problem Set 4 26 September 2024 Prof. Wil Gonzales 17 General designs 26 September 2024 Prof. Wil Gonzales 18 General designs 26 September 2024 Prof. Wil Gonzales 19 Between vs. Within-subjects design (PS 4d) A researcher has a sample of 30 rats that are all cloned from the same source. The 30 rats are genetically identical and have been raised in exactly the same environment since birth. The researcher conducts an experiment, randomly assigning 10 of the clones to treatment A, 10 to treatment B, and the other 10 to treatment C. Explain why the clone experiment is better than both a between- subjects study using 30 regular rats that are randomly assigned to the three treatments and a within-subjects study using 10 regular rats that are tested in each of the three treatments. In other words, explain how the clone experiment eliminates the basic problems with both between-subjects and within-subjects studies. 26 September 2024 Prof. Wil Gonzales 20 Experimental Design: The Role of Random Assignment Random assignment is the process of assigning participants to different groups or conditions in an experiment using random methods (e.g., random number generators, drawing lots). Purpose: Ensures that each participant has an equal chance of being placed in any group, minimizing biases. 26 September 2024 Prof. Wil Gonzales 22 Why is Random Assignment Important? Controls for Confounding Variables: Reduces the influence of external variables (e.g., age, gender, background) by evenly distributing them across groups. Ensures Group Equivalence: Helps ensure that differences in outcomes are due to the independent variable rather than pre-existing differences between groups. Enhances Internal Validity: Strengthens the ability to draw causal inferences by isolating the effect of the independent variable. 26 September 2024 Prof. Wil Gonzales 23 Example of Random Assignment in Action Scenario: Testing a new language learning app. Group 1: Randomly assigned participants using the app. Group 2: Randomly assigned participants using a traditional learning method. Outcome: Any observed differences in language proficiency can be more confidently attributed to the learning method, not pre-existing differences. 26 September 2024 Prof. Wil Gonzales 24 Key Takeaways Random assignment is crucial for maintaining the integrity of experimental research. It helps ensure that the observed effects are genuinely due to the experimental treatment. Without random assignment, the ability to make causal claims is significantly weakened. 26 September 2024 Prof. Wil Gonzales 25 Experiment designs in applied linguistics Pretest-posttest design Control Post- Pre-test group test Experimental Pre-test Intervention Post- group test 26 September 2024 Prof. Wil Gonzales 26 Experiment designs in applied linguistics Time series design Control Pre Pre Pre Post Post Post group 1 2 3 1 2 3 Experimental Pre Pre Pre Post Post Post Intervention group 1 2 3 1 2 3 26 September 2024 Prof. Wil Gonzales 27 Which design would you use? … if you want to examine the effect of TV viewing on the incidental learning of L2 vocabulary? Control Post- Pre-test group test Experimental Pre-test TV viewing Post- group test 26 September 2024 Prof. Wil Gonzales 28 Which design would you use? … if you want investigate the impact of video gaming on development in the use of the Japanese te-i-(ru) construction? Control Pre Pre Pre Post Post Post group 1 2 3 1 2 3 Experimental Pre Pre Pre Post Post Post Video game group 1 2 3 1 2 3 26 September 2024 Prof. Wil Gonzales 29 Latin square design a type of experimental design that controls for order effects and interactions between treatments by ensuring that each treatment appears once in each position. used when studying the effects of multiple treatments (2 or more) on a dependent variable. 26 September 2024 Prof. Wil Gonzales 30 How is it different from other designs? The previous tests: focus on measuring changes within participants over time and observing changes in variables across multiple time points. Latin-square: involve multiple treatments applied to different groups or conditions and focus on controlling order effects and interactions between treatments. 26 September 2024 Prof. Wil Gonzales 31 Latin square design: In simple terms 26 September 2024 Prof. Wil Gonzales 32 Latin Square Design Practice (PS 4f) You are a researcher investigating the effect of three different teaching methods (A, B, and C) on the acquisition of pronunciation skills in a second language. You have a group of 9 students who will each participate in three sessions, where each session will use one of the teaching methods. To ensure that the results are not biased by the order in which the methods are presented, you need to design the study so that each student experiences all three methods in a different order. 26 September 2024 Prof. Wil Gonzales 33 Latin Square Design Practice (PS 4f) Illustrate your solution (Latin Square Design) by completing the table below, where each row represents a student, and each column represents the session. In each cell, indicate which teaching method (A, B, or C) will be applied. 26 September 2024 Prof. Wil Gonzales 34 Repeated measures design within-participants designs — a single group of participants who take part in all the different treatment conditions and/or are measured at multiple times One group Treatment Treatment Treatment Treatment (functions as control) 1 2 3 4 26 September 2024 Prof. Wil Gonzales 35 Repeated measures design each participant is exposed to all levels of the independent variable(s). the same group of participants is tested under different conditions or treatments. GOAL: compare the effects of different conditions within the same group of participants, minimizing individual differences and increasing the study's statistical power. Any foreseeable cons? 26 September 2024 Prof. Wil Gonzales 36 Use this design for… Minimizing Individual Differences because participants serve as their own control group. reduces the impact of individual differences Efficiency require fewer participants compared to between-subjects designs Reducing Noise By using the same group of participants across conditions, researchers can better detect the effects of the independent variable(s) by reducing the influence of extraneous variables 26 September 2024 Prof. Wil Gonzales 37 In relation to other designs… Function Repeated minimize individual differences and measures increase efficiency in experiments Latin square control for order effects and interactions between treatments Pretest-posttest measure changes over time Time series observe variable changes over multiple time points 26 September 2024 Prof. Wil Gonzales 38 Factorial design A design that involves the simultaneous manipulation of two or more independent variables to examine their individual and interactive effects on a dependent variable. 26 September 2024 Prof. Wil Gonzales 39 Factorial design Independent variables = factors the combined effects on the dependent variable Can be between- and within-participants 26 September 2024 Prof. Wil Gonzales 40 Example Research Question: How does the type of feedback (written vs. oral) and the frequency of feedback (weekly vs. monthly) impact second language learners' performance in grammar exercises? Factors/Independent variables: 1. Type of Feedback: This factor has two levels: written feedback and oral feedback. It explores whether the mode of feedback delivery affects learners' grammar performance. 2. Frequency of Feedback: This factor also has two levels: weekly feedback and monthly feedback. It investigates whether the frequency of feedback provision influences learners' grammar improvement. Dependent Variable: Grammar exercise scores. 26 September 2024 Prof. Wil Gonzales 41 Design: 2 by 2 factorial design Written Oral feedback feedback Weekly 10 10 feedback Monthly 10 10 feedback Participants randomly assigned to one of the four conditions. Over a specified period, they receive the assigned type and frequency of feedback while completing a series of grammar exercises. 26 September 2024 Prof. Wil Gonzales 42 Design: 2 by 2 factorial design allows researchers to determine whether the type of feedback, the frequency of feedback, or the interaction between these factors has a significant impact on learners’ grammar exercise scores Main effect Does type influence scores? Does frequency influence scores? Interaction effect Does type depend on frequency to influence scores? (and vice versa?) Please refer to Problem Set 4i and 4j for more practice 26 September 2024 Prof. Wil Gonzales 43 In relation to other designs… Function Factorial study main effects and interaction effects of multiple independent variables on a dependent variables Repeated measures minimize individual differences and increase efficiency in experiments Latin square control for order effects and interactions between treatments Pretest-posttest measure changes over time Time series observe variable changes over multiple time points 26 September 2024 Prof. Wil Gonzales 44 What if my study doesn’t have ALL experimental elements? manipulation of cause-and- variables effect inference control group control of random extraneous assignment variables measurement replicability of effects 26 September 2024 Prof. Wil Gonzales 45 Quasi-experiment! lack some of the key elements of true experimental designs, such as random assignment of participants to different conditions often involve comparing existing groups that were NOT created by the researcher (e.g., different classrooms, schools, or communities). Pretest-posttest but with existing sections often used when it’s not feasible or ethical to implement full experimental control E.g., classroom settings 26 September 2024 Prof. Wil Gonzales 46 Limitations of quasi-experiments Cannot really establish causal relationships Cannot control for some confounding variables Nevertheless: They still provide valuable insights into relationships between variables in real-world contexts!!! 26 September 2024 Prof. Wil Gonzales 47 Problem Set 4h: Be-an-experimenter In the field of applied linguistics, choosing the appropriate research design is crucial for ensuring that your study effectively addresses the research questions and objectives. This worksheet presents several scenarios that require you to make informed decisions about research design. You will need to consider factors such as participant organization, sampling methods, and the specific objectives of each study to propose the most suitable research design. Please refer to Problem Set 4h 26 September 2024 Prof. Wil Gonzales 49 Check out the following for more practice PS 4d2 PS 4e 26 September 2024 Prof. Wil Gonzales 50 The plan… Lecture and problem sets! 1 WEEK 5 Corpus-based method Prof. Wilkinson ‘Wil’ Daniel Wong Gonzales 28 September 2024 Prof. Wil Gonzales 2 Corpus-based methods and corpus linguistics Corpus linguistics: the study of language based on large collections of “real life” language use stored in corpora (Nordquist 2019) The origin of the corpus-based method Key Focus: Variation in language use. Patterns of usage that speakers are often unaware of. Language in context. 28 September 2024 Prof. Wil Gonzales 3 History can be traced back to the 1960s and 1970s, with the development of the Brown Corpus (1961) as one of the first computerized corpora. 1960s-1970s: Emergence of early corpora and the use of computers to analyze language. 1980s-1990s: Growth in the size and variety of corpora; development of more sophisticated analytical tools. 2000s-Present: Integration of corpus methods into mainstream linguistics and applied fields; widespread use of large, annotated corpora like the British National Corpus (BNC) and Corpus of Contemporary American English (COCA). evolved from a niche area to a central methodological approach in many branches of linguistics: led to “corpus-based method” 28 September 2024 Prof. Wil Gonzales 4 What is the “corpus-based method”? involve the systematic analysis of language data (corpora) using computational tools to uncover patterns, frequencies, and structures in real-world language use. can be applied across various linguistic levels, including syntax, semantics, pragmatics, and sociolinguistics. data-driven, often involving large-scale datasets to study language use in authentic contexts. 28 September 2024 Prof. Wil Gonzales 5 Example of corpus-based method 28 September 2024 Prof. Wil Gonzales 6 Components of the corpus-based method (Barth and Schnell 2021) corpora (singular: corpus): a collection of texts that serves as the empirical basis for the study of natural languages. 28 September 2024 Prof. Wil Gonzales 7 Some might say a corpus is…. (Davies 2017) ‘systematically organized data bank’ Searchable collections of spoken and/or written language (nearly always in electronic format) that can be used for (socio)linguistic analysis. Examples of English Corpora ICE-GB corpus structure International Corpus of English – Hong Kong component (Nelson 1990) British National Corpus Corpus of Singapore English Messaging (CoSEM, Gonzales et al 2021) Text corpora! AntConc corpus analysis software 28 September 2024 Prof. Wil Gonzales 8 Components of the corpus-based method (Barth and Schnell 2021) corpora (singular: corpus): a collection of texts that serves as the empirical basis for the study of natural languages. A text is any instance of recorded language use that can be treated as a discrete unit: a newspaper article, a recorded university lecture, or dinner table conversation. Corpora and text must be machine-readable so that they can be collated and investigated with the help of computers. 28 September 2024 Prof. Wil Gonzales 9 Structure of a corpus (Dash & Arulmozi 2018) Metadata 28 September 2024 Prof. Wil Gonzales 10 What comprises “text”? Word-form these are forms that are directly observable and reflect any inflexions appropriate for their context: ‘go’, ‘going’, ‘goes’, ‘went’, ‘women’, ‘woman’, ‘women’s’. Lexeme the abstraction underlying inflectionally related groups of wordforms that share lexical meaning; often one wordform is used to represent a lexeme, for example, GO or WOMAN, as the head word in a dictionary. The all-caps writing is intended to indicate that this is the name of an abstract unit. 28 September 2024 Prof. Wil Gonzales 11 Units of analysis Texts consist of wordforms, and wordforms enter larger abstract structures. 28 September 2024 Prof. Wil Gonzales 12 Another corpus-based method essential… Types: Unique words in a corpus; Tokens: Total instances of words in a corpus; individual wordforms in a text. Example Sentence: "Apples are red. Red apples are sweet. Apples are delicious." How many types? How many tokens? 28 September 2024 Prof. Wil Gonzales 13 Another corpus-based method essential… Collocates words that frequently appear together or in close proximity within a given text or corpus. co-occur more often than would be expected by chance. provide insights into the typical ways words are used in context can reveal patterns of meaning, usage, and relationships between words. 28 September 2024 Prof. Wil Gonzales 16 What’s the point of studying corpora? What can they help us uncover? linguistic patterns and potential language change WEAKNESSES STRENGTHS Typical lack of linguistic Data ‘in the wild’ metadata (need to manually or Quantity computationally link) Quality of data 28 September 2024 Prof. Wil Gonzales 17 What exactly characterizes a corpus-based approach? Empirical analysis: actual patterns of use in natural texts Large and principled collection of natural texts (corpus) The use of computers of analysis Quantitative and qualitative techniques (Biber et al 1998:4) 28 September 2024 Prof. Wil Gonzales 18 Example of quantitative study: Buson et al 2023 28 September 2024 Prof. Wil Gonzales 19 Example of qualitative study: Tsoumou 2023 28 September 2024 Prof. Wil Gonzales 20 Example of qualitative study: Tsoumou 2023 28 September 2024 Prof. Wil Gonzales 21 There are plenty of corpora to choose from… You are tasked to analyze the frequency distribution of the use of split infinitives in English. e.g., to boldly go, to really say Which corpus would you choose? 100-million-word 500k-word corpus corpus More structured Less structured Less noise (URLs, More noise (URLs, emojis) emojis) 28 September 2024 Prof. Wil Gonzales 22 Bigger is not always better the size of a corpus required for a particular study always depends on the specific research question(s) and the level of granularity that the analyses aim to provide small and carefully compiled corpora may therefore be complemented with large (often web-based) databases 28 September 2024 Prof. Wil Gonzales 23 Also: limitations of corpus-based approaches lack of metadata Regional information Age, etc. potentially noisy data “mainly targeted towards the study of high-frequency phenomena at the syntactic level” If you are investigating the ‘rare’ phenomenon of split infinitives (e.g., to NOT go) in a English corpus, and do not find instances of split infinitives. Does it mean that split infinitives are ungrammatical? 28 September 2024 Prof. Wil Gonzales 24 What Schütze and Sprouse (2013) say… a feature’s “failure to appear in even a very large corpus (such as the Web) is not evidence for ungrammaticality, nor is appearance evidence for grammaticality” 28 September 2024 Prof. Wil Gonzales 25 So do we just abandon corpus-based methods? No! Corpus-based methods are still useful! Key idea: triangulation and use of complementary methods to answer questions you have One such method: Experiments! 28 September 2024 Prof. Wil Gonzales 26 Going back to corpora: What does it mean for a corpus to be ‘principled’? Representative of the “population” o Is it possible to do this? o E.g., people use spoken language more than written language, but we have way more text corpora compared to speech corpora o Solution: judgment sampling o Sample more speech data, for example o Still principled! 28 September 2024 Prof. Wil Gonzales 27 Popular corpora that adopts judgment sampling International Corpus of English More spoken than written 28 September 2024 Prof. Wil Gonzales 28 Web-as-corpus approach: representative? GloWbe and NOW corpora provides useful evidence on a low-frequency phenomenon that supplement findings on standard reference corpora 28 September 2024 Prof. Wil Gonzales 29 Is there such a thing as “outdated” corpora? Should linguists use these “[n]ew resources or just better old ones?” There’s still mileage! 1. Study replication with new methodologies 2. Diachronic/historical comparison 3. Easy and “pleasant” to use because of structure learned learnt 28 September 2024 Prof. Wil Gonzales 30 Key ideas Corpus-based approach can inform your methodology. Entails quantitative OR qualitative empirical analysis of large and principled collection of texts using computers. Bigger is not always better; there is no “best” corpus. Each corpus has its own uses Ensure that your data answers your research questions. 28 September 2024 Prof. Wil Gonzales 31  Corpus-based methods in Applied Linguistics 28 September 2024 Prof. Wil Gonzales 32 Why are corpus-based methods valuable for Applied Linguistics? Can help answer questions like these… How do learners acquire specific language structures or vocabulary? What language features are most challenging for second language learners? How can language use in textbooks be aligned with authentic language use in real-life contexts? How do social and cultural factors influence language use in different contexts? 28 September 2024 Prof. Wil Gonzales 33 Why are corpus-based methods valuable for Applied Linguistics? Reflection of Real-World Usage: Corpus-based methods analyze large collections of real texts, offering an accurate picture of how language is actually used. Authenticity is crucial for developing language teaching materials, designing language policies, and understanding language variation. 28 September 2024 Prof. Wil Gonzales 34 Why are corpus-based methods relevant to Applied Linguistics? Objective Analysis By focusing on empirical data, corpus-based methods minimize the subjectivity that can come from self- reported data or experimental settings allows for more reliable conclusions about language patterns, trends, and norms, which can inform language education, policy-making, and communication strategies. 28 September 2024 Prof. Wil Gonzales 35 Why are corpus-based methods relevant to Applied Linguistics? Solving Practical Problems Language and Technology: Corpus-based methods are often used in the development of language technologies, such as spell checkers, grammar checkers, and translation tools Policy Development: By providing evidence of how language is used in various domains, corpus data can inform language policies that are more in tune with the needs of specific communities, such as in multilingual education or workplace communication. 28 September 2024 Prof. Wil Gonzales 36 Usefulness of Corpus-Based Methods in Applied Linguistics Studying Language Variation: Examining how language varies across different social groups, regions, or registers. Lexical Studies: Analyzing word frequency and collocations to understand vocabulary use in context. Language Teaching: Designing teaching materials based on authentic language use. Discourse Analysis: Understanding how language is used in different types of texts (e.g., political speeches, academic writing). 28 September 2024 Prof. Wil Gonzales 37 Possible projects Example 1: Investigating the use of modal verbs in academic vs. spoken English. Example 2: Analyzing collocations in learner English to identify common errors. Example 3: Studying discourse markers in conversational English to understand spoken language structure. 28 September 2024 Prof. Wil Gonzales 38 In relation to other methods we discussed Corpus-Based Experimental Aspect Survey Methods Methods Methods Data Source Real-world texts Participant Controlled responses experiments Focus Frequency, Perceptions, Hypothesis testing, collocations, attitudes cause-effect language patterns Strength Reflects actual Insight into Identifying causal language use language attitudes relationships and beliefs 28 September 2024 Prof. Wil Gonzales 39 Key points Corpus-based methods offer a powerful tool for studying real- world language use in applied linguistics, complementing survey and experimental approaches. The choice of method depends on research goals—corpus methods excel in studying authentic language patterns, while surveys and experiments are bett