Research Design and Methodology PDF

## Research Design and Methodology In this chapter, research design and methodology is explained within the domains of IT. ### Typical Research Question * How many? How frequently? What percentage? What proportion? To what extent? * The outcome is usually a set of numeric values that can be stati...

## Research Design and Methodology In this chapter, research design and methodology is explained within the domains of IT. ### Typical Research Question * How many? How frequently? What percentage? What proportion? To what extent? * The outcome is usually a set of numeric values that can be statistically analyzed to draw inferences and comparisons. #### Example In computer science, it might be a study where researchers are testing the effectiveness of a new algorithm. For Example: * **Empirical Observation:** Researchers could empirically observe the performance of the algorithm in a controlled environment or in real-world applications. * **Objective Measurement:** They would collect data on its efficiency, accuracy, and speed, compared to existing algorithms. * **Independent of Researcher's Mind:** The results of these test would be considered valid irrespective of the researchers' preconceived notions or expectations about the algorithm's performance. * **Scientific Theories and Concepts:** They would use established scientific concepts and metrics to interpret the data and to express the nature of the algorithm's performance. * **Replicability:** Other researchers should be able to replicate the experiment under the same conditions and achieve the same results, which is a key principle. * So, in a computer science context, it involves conducting research in a manner that is quantifiable, observable, and verifiable by others. ### Methodologies * **Methodologies:** Incorporates a range of methodologies, tools, and techniques aimed a understanding social reality, including how organizations function. * **Understanding Social Phenomena:** Researchers interpret the social aspects of Phenomena by reviewing documents like public policies, meeting minutes, and institutional rules. * **Data Collection Through Interviews:** Interviews are conducted to gather data from individuals who are closely acquainted with the Phenomenon being studied. * **Analysis Shaped by Experiences:** The Analysis and conclusions in constructivist research are heavily influenced by the participants’ accounts of their lived experiences. ### Typical Interpretive Research Questions * The interpretivist usually applies qualitative thinking. * The requirement is to: * "discover", "generate", "explore", "identify", or "describe" events in an exploratory manner. * Qualitative research questions generally start with: * "how", "why", "in which ways", "to what extent" #### Example In computer science, interpretivism might be applied in the study of how people interact with technology or how organizational culture affects the adoption of new IT systems. Here's an example: * **Study on User Experience** (UX): A researcher might want to understand how users perceive and interact with a new software interface. Instead of just measuring click rates or error rates (as in positivist research), an interpretivist study would delve into users' subjective experiences. * **Qualitative Data Collection:** The researcher conducts in-depth interviews or focus groups with users to gather rich, qualitative data about their experiences, thoughts, and feelings regarding the software. * **Contextual Factors:** The study would consider the social and cultural context of the users, such as how their backgrounds or job roles influence their interaction with the software. * **Data Interpretation:** Instead of seeking objective truths, the researcher interprets the data to provide insights into how different users uniquely perceive and construct the meaning of the software's usability. ### Pragmatism * **Problem-Oriented Approach:** Pragmatism in research is centered around understanding and solving specific problems. * **Researchers using pragmatism** are not restricted to any one methodology; they choose whatever methods are necessary to gain understanding. * **Researchers may use several methods** either from the quantitative realm (like surveys and experiments) or qualitative realm (like interviews and observations). ### Typical Pragmatic Research Question 1. What can be done to increase literacy of adult learners? 2. Does ODL (Online Distance Learning) increase student satisfaction and completion rates? 3. How to increase software developers satisfactions in an organization? 4. What incentives are effective for encouraging software developers to be more productive? * It focuses on practical applied research: 'what works'. * It focuses on solving the problem. #### Example In computer science, an example of pragmatism could be a study that aims to improve user experience on a social media platform: * **Identifying the Problem:** The researcher begins by identifying a problem, such as a decline in user engagement on the platform. * **Quantitative Methods:** Initially, a quantitative survey might be conducted to gather data on how frequently users engage with different features of the platform. * **Qualitative Methods:** Subsequently, qualitative interviews or forum groups might be used to delve deeper into the reasons behind users' preferences or aversions to certain features. * **Iterative Development:** Based on these findings, the researcher might develop a prototype of a new feature and then use quantitative measures to test its effectiveness, followed by qualitative feedback for refinement. * **Outcome Focused:** The ultimate goal of the research is to create a tangible improvement in user engagement, rather than to confirm a specific theoretical perspective. * **Adaptive Methods:** Throughout the process, the researcher remains open to using new methods or altering the research design based on what is most practical for solving the problem at hand. ### Realism * **Realism in research focuses on** understanding the true nature of what is real with respect to the research problem. * **Realists find the data from** positivist (quantitative) and interpretivist (qualitative) approaches to be insufficient for a deep understanding of research problems. * **Realists believe that even the pragmatic combination of** methodologies does not go far enough in depth to understand the research problem fully. * **Realists often select constructivist case study methods** to gain a deep insight into a problem. * **Realists aim to uncover the truth about the world** by studying the properties and processes of things as they exist in reality. #### Sample Critical Research (Realism) Questions * Why does Facebook own all the content that we supply? * Does the power of the net further marginalize the non-connected? * Who benefits from data disclosure? * Why did the One Laptop Per Child fail? * Does learning analytics exploit student vulnerabilities and right to privacy? * Are MOOCs (Massive open online course) really free? * Who owns and for what use are learning analytics? * Does Online education expose learners to more educational failure? * Realism is regarded as metaphysical (aiming to understand the relationship between the mind and the physical universe). #### Example In the context of computer science, a realist might be interested in studying the actual performance and effects of a new artificial intelligence (AI) Algorithm in practical, real-world settings. Here's how realism could be applied: * **Study on AI Algorithm Performance:** A computer science researcher may want to understand the true performance of an AI algorithm in various conditions. * **Objective Measurement:** The realist would objectively measure the performance of the algorithm, not just under ideal laboratory conditions but also in real-world applications where variables are less controlled. * **In-depth Case Studies:** The researcher might conduct in-depth case studies on how the AI algorithm interacts with other systems and affects them, aiming to reveal a comprehensive understanding of the AI's functionality and its implications. * **Real-world Applications:** For instance, studying how an AI system for automated customer service affects actual customer satisfaction and company workflows in a real business environment. ### Methodology * **Definition of Methodology:** Methodology refers to the nature and structure of the research process, encompassing how specific methods, principles, and techniques are applied in a study. * **Research Process Steps:** It involves identifying, selecting, collecting, processing, analyzing data, and drawing conclusions in a particular field of study. * **Consideration in Method Selection:** When choosing a methodology, researchers must consider the nature of the research problem to decide whether a quantitative, qualitative, or mixed-methods approach is most suitable for data collection and analysis. ### Research Design * **Research design provides a structured plan** detailing the steps necessary to generate knowledge, including defining the research problem, hypotheses, and analytical framework. * **It involves selecting appropriate methods** for data collection, presentation, and analysis. * **Inclusion of Methodological Approaches:** The design encompasses action plans for qualitative, quantitative, or mixed-methods approaches. #### Foundational Elements of Research Design and Methodology * Research design can be represented visually, as exemplified by the generic diagram shown in figure * The next figure shows five elements of research design that can guide a researcher in the fields IT. #### Conceptualization of elements of research design [Diagram of a circle with five sections, connected with arrows inside the circle: Context, Philosophy, Methodology, Research Design, Methods(data)] #### IT Methodology Thinking Framework [Diagram of a diamond-shaped figure with four sections, connected with arrows inside the figure: Philosophical Assumptions, Methodology, Research methods, Research Design] ### The Goal * The goal is to present a proposal that ensures that the **research design is suitable to obtain valid** objective/subjective and meaningful answers to the research questions since research is about creating meaning from the relative chaos of ideas and data. * **Only sound research design can ensure that the** evidence-gathering will address the research problem effectively, in an unambiguous way. Reviewers and examiners take great care in considering the quality of research design and its effectiveness in addressing the research problem. ### Steps for Research Design * **Step 1:** Since you have identified the research problem clearly. * **Step 2:** If the research philosophy is positivist, then decide what type of positivism applies, for example descriptive, correlational, causal comparative, experimental, or quasi-experimental. * **Step 3:** Selecting a research philosophy will help to clarify the research methodology: If you chose a positivist research philosophy, then the methodology will be quantitative or mixed methods. If you chose an interpretivist research philosophy, then the methodology will be qualitative. If you chose a pragmatist research philosophy or a realist philosophy, then the methodology will be qualitative or mixed methods. * **Step 4:** The next step is to clarify the detail of the research methods, deciding whether to adopt a quantitative survey, a design and creation method, a qualitative survey, a case study, or action research. * **Step 5:** With respect to data collection methods, descriptive research may adopt a quantitative survey design, while a constructivist case study design could include interviews, focus groups, documentary review, and observation (or some of these). * **Step 6:** The final step in research design will include deciding on the methods for evaluating the design (Oates, 2006) prior to submission to critical readers/assessors. At this juncture, the researcher should appreciate that the research problem determines the nature of the research design and not vice versa. The understanding and knowledge of the problem is central to the choice of the approaches to data collection and data analysis. ### Types of Research Design * The section primarily discusses research designs commonly used in IT research. 1. Descriptive Research Design 2. Experimental Design 3. Comparative Study Design 4. Causal -Comparative/Quasi-Experimental Design 5. Correlational Research Design * For more details read: * McCombes, S. & Bhandari, P. (2021-2022). Research design: A step-by-step guide with examples. {https://www.scribbr.com/methodology/research-design/} ### Descriptive Research Design * **Purpose of Descriptive Research Design:** This design is used to answer basic questions about who, what, when, where, and how, helping to describe the nature and characteristics of a specific phenomenon. * **Information Gathering:** It enables researchers to collect data on the status of the Phenomenon, focusing on specific variables or circumstances. * **Limitation in Explaining Causes:** While descriptive research provides detailed Information, it does not offer insights into why a Phenomenon occurs. * **Foundation for Further Research:** This type of research is often a preliminary step that informs further quantitative research by identifying important variables for hypothesis testing. * **Examples in Cybersecurity:** In cybersecurity, descriptive research might involve studying organizational attitudes towards global cybercrimes or examining the insider factor in cybersecurity breaches. * **Example in Computer Science:** An application in computer science could be analyzing how different processors affect computer performance. ### Experimental Design * **Experimental design allows researchers to** control variables that might affect the outcome of an experiment. * **Establishing Cause-and-Effect:** It employs **statistical methods to determine the cause-and-effect relationship between** variables in a study. * **Suitability for Studying Relationships:** This design is **ideal for examining the likelihood that independent variables consistently affect dependent variables,** thereby assessing the strength of the relationship. * **Not Limited to Lab Settings:** While often associated with **laboratory work, a true experiment can occur outside a lab setting.** It's characterized by the manipulation and control of independent variables to observe their effects. * **Real-world Experimentation:** In practical applications, **experiments must involve controlled, randomized, and manipulated measurements to ensure accuracy and reliability.** #### Examples of Experimental Studies: * Some examples of experimental study may include the following: * The effect of AI tools on software development. * The effect of malware attacks on individuals. * The effect of artificial intelligence on supporting customers in a retailing shop. * It can be used to substantiate algorithm theories, network theories, memory performance assumptions, AI theories, cyber risk theories, user experience theories, etc. #### Experimental Design Has the Following Advantages: * It helps the researcher to set the limits of the experiment by scoping the boundaries of the study using independent variables. * The researcher is in full control of the independent variables, which allows the researcher to address the question, "what causes an event to occur?". * It helps the researcher to determine the dependent variables based on the independent variables. * This design allows the researcher to determine cause-and-effect relationships between variable factors and to differentiate dummy effects from actual effects. * It enables the researcher to determine what has happened (deductive analysis) or predict what may happen (predictive analytics). * and analysis processes.