Week 3 Lecture Part 2: Statistical Concepts PHCM2100 2024 PDF

Martin R Lindley Week 3 Lecture part 2 Stats (again) issues when evaluating research PHMC2001 2024 Content of part 2 Probability Meaningfulness (effect size) Power Using information in the context of the s...

Martin R Lindley Week 3 Lecture part 2 Stats (again) issues when evaluating research PHMC2001 2024 Content of part 2 Probability Meaningfulness (effect size) Power Using information in the context of the study Probability Equally likely events: the chances of one event are the same as the chances of another event (e.g., heads/tails, roll of a die) Relative frequency: the observed frequency of an event among other events (e.g., when you toss a coin 100 times, you get 48 heads and 52 tails). If events are equally likely, relative frequencies will “pile up” around the theoretical probability (e.g. 50:50 for coin toss)  Interpreting Stats Probability – Alpha: level of chance occurrence (type I error) Typical: alpha =.05 (p <.05) or alpha =.01 (p <.01) Varying alpha: What should we consider when we set alpha? Risks/benefits of a type I error Interpreting Stats ? Exact probability – Beta (type II error) Meaningfulness (determined after considering effect size and relative cost/benefits) Power: probability of rejecting the null hypothesis when it is false (i.e., correctly rejecting the null hypothesis) Testing the Hypothesis Null hypothesis: no difference or no relationship Alternative hypothesis: what you actually expect to happen (a difference or a relationship) Accept / Reject Null Hypothesis Critical Value Prior to conducting the statistical test, you must identify the critical value The critical value is then compared with the test statistic you calculate If the absolute value of the test statistic is larger than the absolute value of the critical value, you reject the null hypothesis and entertain the alternative hypothesis  Alternative Hypothesis We entertain the alternative for two reasons: From a statistical standpoint, there is always the chance of a type I error. Even if alpha = 0.01, there is still 1 chance out of 100 that our conclusion will be incorrect. If you conduct an experiment again, but with a different sample, results may differ. Alternative Hypothesis What should be in a paper What should be included when reporting the results of a statistical analysis The value of the test statistic The exact probability of obtaining that test statistic due to chance (or simply indicate whether p is < or > alpha) The effect size Effect Size Effect size (ES): the standardized difference between two means. Formula: ES = (M1 – M2) / s ES allows comparison between studies using different dependent variables because it puts data in standard deviation units. An effect size of 0 is no difference, 0.2 is small, 0.5 medium, and 0.8 large. Always context How do findings from the study fit within the context of  Theory  Practice When you plan your research Information needed in planning  Alpha  Effect size  Power  Sample size Reading a research paper When reading research, often sample size, means, and standard deviations are supplied. You can calculate the effect size of the research. Using these data and the Power Calculator at the Web site below or G*Power, you can estimate the power to detect a difference or relationship. www.stat.ubc.ca/~rollin/stats/ssize/n2.html Think about things Power, alpha, and sample size all have an influence on whether or not you will find a significant result. If sample size is extremely large, even a very small effect will be judged significant. If sample size is very small, extreme scores can dramatically influence the results. Finding statistical significance is unlikely for small and moderate effects.  When you do research How was power analysis done? Always report complications (screen your data). Select statistical analyses. Report p values of confidence intervals. Report magnitudes of the effects. Control multiple comparisons. Report variability using standard deviations. Report data to appropriate level. Week 3 – Tutorial Read the new journal article prior to attending your scheduled class. NB – 80% attendance is required to complete the class. Course convenor Dr Lindley [email protected]  Martin R Lindley Week 3 Lecture part 1 Statistical Concepts PHCM2100 2024 Content of part 1  Why we need statistics  Understanding description, inference, and statistical techniques  Ways to select a sample  Ways to assign participants to groups  Measures of central tendency and variability  Basic concepts of statistical techniques Types of Statistics  Descriptive techniques: measures of central tendency and of variance  Correlational techniques: to determine the relationship between two or more variables  Differences between groups: to determine if there is a reliable difference between the means of two or more groups  Statistical Tests All statistical tests are based on the same general equations: To test for differences between groups To test for relationships Why Stats ? Statistics are an objective way of interpreting a collection of observations. Statistics allow us to establish the significance and the magnitude of an effect. Types of statistics  Descriptive techniques  Correlational techniques  Differences among groups Correlation - Pearsons The Pearson product moment coefficient of correlation (Pearson’s) is used to describe a linear relationship between two continuous variables (X and Y). The symbol is r and this ranges from  −1 (a strong negative correlation) to  +1 (a strong positive correlation). By calculating r2 we can know the percent of variance in Y that is explained by X Correlation - Pearsons What is the nature of this relationship? Which variable is likely considered the independent variable? Which variable is the dependent variable? Software for Statistics Popular software for statistics  Simple spreadsheets (Excel etc)  Statistical Analysis System (SAS)  Statistical Package for the Social Sciences (SPSS)  Stata  R: free statistical tool gaining in popularity  Description and Inference Statistical techniques are used to analyze data from samples and consist of descriptive and inferential techniques.  Descriptive statistics merely “describe” the sample (mean, standard deviation).  Inferential statistics allow for inferences from a sample to a population when the sample represents the population. Samples  Random sampling  Stratified random sampling  Voluntary sample  Convenience sampling  Snowball sampling Which of these is most commonly used? Which of these is least commonly used? Assigning participants  Random assignment  Random matched assignment  Intact groups How are post-hoc justifications used to support sampling and participant assignment to groups? Sampling How good does it have to be? To answer this question, we need to consider what the population is that we’re trying to generalize to and the extent to which the sample represents the population. Variability – Central Tendency Central tendency scores  Mean: average (µ for population, M for sample)  Median: midpoint  Mode: most frequent Variability scores  Range of scores  Standard deviation (σ for population, s for sample)  Standard Error of Measurement (SEM) For data that is normally distributed - mean and standard deviation. Confidence Intervals A confidence interval (CI) provides the expected upper and lower limit for a statistic (e.g., the mean). When we use a sample to estimate the mean, we will have sampling error (i.e., the sample will not be exactly the same as any other sample or as the population) The CI is an interval that is 95% likely to contain the true mean. Types of Statistical Tests Parametric  Normal distribution  Equal variances  Independent observations Nonparametric (distribution free)  Distribution is not normal Normal Distribution Research Hypotheses Research hypotheses  Expected results based on theory or experience  Stated as outcomes  Testable Null hypotheses No statistically significant differences or relationships  Statistics What statistical techniques tell us  Reliability (significance) of effect  Strength or magnitude of the relationship Types of statistical techniques  Relationships among variables  Differences among groups Cause and effect: Correlation is no proof of causation. Week 3 – Tutorial Read the new journal article prior to attending your scheduled class. NB – 80% attendance is required to complete the class. Course convenor Dr Lindley [email protected]  Martin R Lindley Week 2 Additional notes Variables/Data PHCM2100 2024 Types of Data - Variables you will come across in research papers. Categorical Variables  You are working with categorical data if you cannot add them together … Variable A – Dogs Variable B – Cats ….. You can’t add them together Similarly your grades in school A ,B, C etc ….you can’t add them up. (you would have to use numbers 60%etc)  Ordinal Variables  You are working with ordinal data if they have separate categories and can also be ranked in order. Back to your grades in school A B C etc ….. A is a higher grade than a B and so these letter grades are ordinal. Similarly for Economic status of research participants we have High, Medium or Low economic status. They are separate categories that cannot be added but can be ranked. Nominal Variables  You are working with nominal data if they are separate categories but cannot be put into rank order. You will see marital status recorded in research papers. Married or Single. One is not ranked above the other and they are separate categories. If you add another separate category eg civil partnership this does not change.  Numerical Variables  You are working with numerical data if they represent a measurable quantity. One of the primary characteristics of numerical data is that numerical variables can be subjected to mathematical operations, such as addition, subtraction, multiplication, and division. Age and annual income would both be numerical variables.  Continuous Variables  You are working with continuous numerical data if they are a measurable quantity which can take on an infinite number of values across a range. Examples of continuous variables include height, weight, and temperature. (yes there are ‘physical’ limits but this is a conceptual framework for the definition of data).  Discrete Variables  You are working with discrete numerical data if they are a measurable quantity which can take on a finite number of distinct values. The number of students in your tutorial room is a discrete numerical data point. There is a capacity limit to the room. Read the new journal article prior to attending your scheduled class. NB – 80% attendance is required to complete the class. Course convenor Dr Lindley [email protected]  Martin R Lindley Week 2 Lecture part 2 Formulating the method PHCM2100 2024 Content of part 2  How to present methodological details  Why planning the methods is important  Two principles for planning experiments  Describing participants  Selecting and describing instruments Content of part 2….continued  Describing procedures  Describing design and analysis  Establishing cause and effect  Interaction of participants, measurements, and treatments  Presenting method/procedure  Research articles versus thesis or dissertation A thesis will present more details on the methods than a research article In journal format those details are placed in the appendix  Research articles Conserve space Assume the author ‘knows’ Rely on citations to original “methods”  The Methods section  Parts of the methods section Participants Instruments or apparatus or equipment or tools Procedures Design and analysis Basic principles of planning Cohen 1990  Two principles of planning Less is more Simple is better  Are all the independent and dependent variables necessary?  Can you understand and interpret the results? Describing Participants  Special characteristics: What relates to the study? Age and sex/gender Training level/health indices  What to tell about participants Number (included) Loss of participants (excluded versus drop out)  Protecting participants (see Ethics) Describing Participants…. Successful selection of participants  Is based on selection criteria aligned with the purpose of the study  Includes enough range (variation) to produce or identify differences or relationships Describing Equipment  Questions to consider in selecting instruments Validity and reliability Difficulty of obtaining measures Access to equipment or tests Knowing how to use them  What should be presented Description (including validity and reliability) Drawings, photographs, sample items Scoring method The instruments must detect a range of performances  Describing Procedures  What will happen When, where, how much time Pilot data: Can you do this? Scheme for data acquisition, recording, and scoring  Planning treatments How long, how intense, how often Participant adherence Pilot data: Can participants do this? Appropriate treatment of participants  Describing Procedures….  Details of the procedures Specific order of things Timing of events Instructions given Briefings, debriefings, safeguards  What will go in the appendix (research article format)?  Procedures should be consistent with no range or variation (expect between treatments) Avoiding faults in the method ?  Piloting your procedures Can you do this? Can participants do this? Do measures work? Do treatments work? Avoiding faults in the method …..?  Pilot data is typically presented at a supervisor meeting  Pilot data and the methods section can be shared with your supervisors before the proposal meeting  Placing post hoc blame on the methodology for inadequate results is unacceptable Design and Analysis Design is critical The goal is to design a quasi-experimental or experimental study in which The treatments cause the changes observed The variables are related with no intervening variables Design and Analysis….  Analyzing the data Correct analysis Correct interpretation  Establishing cause and effect Independent variable is responsible for changes in the dependent variable Influence Interactions of Participants and measurements Participants and treatments Measurements and treatments Participants, measurements, and treatments Influence  Experimental and quasi-experimental studies Explore the relationship between variables Explore the impact of a treatment on the dependent variable  Statistical analysis for these studies requires variability or range of scores Thus, participants and instruments must capture a range of performance Week 2 – Tutorial Read the new journal article prior to attending your scheduled class. NB – 80% attendance is required to complete the class. Course convenor Dr Lindley [email protected]  Martin R Lindley Week 2 Lecture part 1 Presenting the Problem PHCM2100 2024 Content of wk 1 part 1  Choosing the title  Writing the introduction: background and justification  Stating the research purpose  Presenting the research hypothesis  Operationally defining terms  Basic assumptions, delimitations, and limitations  Justifying the significance of the study  Differences between the thesis and the research article Order versus Practicality Previous slide presents the parts of a research article in the order these appear on paper You do not have to write them in order! No one knows (except the authors) Things will change—like the title  A Goldilocks Title Not too long, not too short—just right The title must  Succinctly state the study’s content  Define the purpose of the study  Capture the reader’s interest  Be easily indexed  Have meaning for the audience Background / Justification  Create interest in the purpose  Persuade the reader the problem is significant  Provide background information  Bring out areas of needed research  Logically proceed with the purpose Steps to an Introduction  Read, review, and understand the literature (previous week)  Determine the problem and hypotheses  Know the audience  Write the background and justification (introduction)  Omit technical jargon (Then write a working title !!) Helpful hints The background should lead the reader  A story that begins “it was a dark and scary night” captures the reader  The reader knows what is coming Draw the reader in  Clear concise writing (not jargon)  The reader should predict the purpose before reading the purpose  Statement of the Problem/Purpose  The purpose statement should be a single sentence that describes the problem  This has previously called statement of the problem  Whats the purpose ? While physical fitness in children, adolescents, and adults has been promoted, little is known about the effects of maintaining motor skills on physical fitness throughout the lifespan. Intermediate to high levels of competence in fundamental motor skills required for successful participation in many sports. Physical activities may be associated with higher levels of performance and health-related physical fitness. Overall, research on the relationship between motor skill competence and aspects of physical fitness is scanty. Stating the Research Purpose Statement of research purpose follows the introductory paragraph  Clear and succinct  One sentence In a traditional thesis format the purpose is before the literature review and after the background (introduction) Identifying the Variables  Independent  Dependent  Categorical (moderator)  Control  Extraneous variables Independent Variables The independent variable is the experimental, or treatment, variable; it is the cause.  Aerobics versus jogging  High versus low intensity  Drug or placebo  Diet or supplement Dependent Variables The dependent variable is what is measured to assess the effects of the independent variable; it is the effect.  Percent body fat  Knowledge  Stature  Anxiety Categorical Variables A categorical variable is a kind of independent variable except that it cannot be manipulated.  Age  Race  Sex Control Variables A factor that could possibly influence the results and that is kept out of the study.  Participants might be sequestered  Time of day  Day of week  Time since you woke up  When you last eat or drank (post prandial)  Medication  Exercise Extraneous Variables A factor that could affect the relationship between the independent and dependent variables but that is not included or controlled. Go back to controlled variables and reevaluate. Research Hypotheses Research hypotheses  Expected results based on theory or experience  Stated as outcomes  Testable Null hypotheses No statistically significant differences or relationships Research Hypotheses These make things clear; Operational definitions  Key terms with specific meaning Limitations  Possible shortcomings  Including delimitations Delimitations  Characteristics imposed by the researcher Published papers Do not always include Operational definitions, limitations, and delimitations Specific research hypotheses Why?  Page costs  The assumption that the authors know more  The assumption that the audience knows Week 2 – Tutorial Read the new journal article prior to attending your scheduled class. NB – 80% attendance is required to complete the class. Course convenor Dr Lindley [email protected]  Martin R Lindley Week 2 Reflections and learnings PHCM2100 2024 Need to reinforce this as IMPORTANT Read read and Review the paper. Take notes. Critique. read again. Approach it from a Record your thoughts Is it strong is it Time spent reading different mindset each about the paper and weak. the paper is always time you read it. What then revisit them next Why do you think well spent. are you looking for ? read through. this ! Variables-Hypothesis-Aim….. Repeat – copy the content Summarise – select the important content Critique – explain why it is important. Its not about good or bad but more how strong is the paper. Work through strengths and weaknesses Read the journal article prior to attending your scheduled class. Engage in the tutorial process. NB – 80% attendance is required to complete the class. Course convenor Dr Lindley [email protected]  Martin R Lindley Week 1 Lecture part 2 Developing the Problem and Using the Literature PHCM2100 2024 Content of part 2  Identifying the research problem  Purpose of the literature review  Basic literature search strategies  Steps in the literature search Find a subject that interests you  Become aware of research being done at your institution.  Look for controversial topics and talk with faculty and other students.  Read a review paper and papers cited in the review.  Make a list of topics or problems.  Review your list of ‘research topics’  Is there a problem that particularly interests you?  Do you have or can you learn the techniques necessary?  Is the problem practical in terms of time, cost, and equipment?  Is there potential to examine cause?  Is the problem meaningful and worthwhile?  Does the problem interest others ? A good idea to try …. Get into groups of 3 to 4 by area of research interest and answer these questions Library  How does the electronic catalog work?  What is the “Web of Science”? - What areas does it cover? - How often is it updated? - How do you use it? Selecting a Research Problem  Is the problem in the realm of research?  Does it interest you?  Does it possess unity?  Is it worthwhile?  Is it feasible?  Is it timely?  Can you attack the problem without prejudice?  Are you prepared in the techniques to address the problem? Inducive / Deductive reasoning Detective Sherlock Holmes uses inductive reasoning.  He was an expert at observing.  He combined his observations into hypotheses.  He tested the hypotheses.  Then he solved the crime! He did not use deductive reasoning, as is often stated. Observation to theory Deductive reasoning Not-so-famous Detective Inspector Lindley often uses deductive reasoning.  He has a suspect in mind.  He examines the evidence with the goal of proving the suspect committed this crime.  Unfortunately, he misses important information and fails to solve the crime. Deductive reasoning There is nothing wrong with starting with a theory.  Starting with a theory and carefully, systematically testing the theory has led to advances in science.  The theory is revised and often extended as study continues.  Schmidt’s schema theory is an example of developing a theory by using deductive reasoning.  Schema theory was a response to Adam’s closed loop theory. Review of the Literature  Identifying the problem  Developing hypotheses  Developing the methods Literature Search 1-6 1. Write the problem statement; this will narrow (or broaden) as your literature search continues. 2. Consult secondary sources. Research reviews (Annual Review of Medicine, etc.) Textbooks 3. Determine descriptors or key words. Typically presented in primary sources Literature Search 1-6 4. Conduct computer-aided searches Use three or more databases to find primary sources Abstracts, Indexes, PubMed, Web of Science, etc. Narrow or expand the search by using Boolean operators Literature Search 1-6 5. Read and record the literature. Primary sources Have a system to record information Include the complete and correct citation Literature Search 1-6 6. Write the literature review in three parts  Introduction explains the purpose and organization of the review  The body is organized around topics Not an annotated bibliography Use headings based on your outline Identify critical versus related studies  The summary and conclusions focus on implications and future recommendations Week 1 – Tutorial Read the journal article prior to attending your scheduled class. NB – 80% attendance is required to complete the class. Course convenor Dr Lindley [email protected]  Martin R Lindley Week 1 Lecture part 1 Introduction to Research PHCM2100 2024 Content of part 1 The nature of research Unscientific versus scientific methods of problem solving Alternative models of research Types of research Overview of the research process Research Characteristics Systematic Plan and identify variables Design to test relationships Collect data Evaluate relationships Logical Examine procedures to evaluate conclusions Research Characteristics…… Empirical Decisions are based on data Reductive General relationships are established from data Replicable Actions are recorded Basic and Applied Research Level I—basic research  Goal: theory-driven  Approach: laboratory Level II—moderate relevance  Goal: theory-based using relevant activities  Approach: similar to real-world task or setting Level III—applied research  Goal: immediate solutions  Approach: real-world settings Ecological Validity Treatment: jogging, aerobic dance and control group Dependent variable: bioelectric impedance Ecological validity  Does the participant perceive the research as intended?  Is the setting real-world enough to generalize? Research – where does it start ? Research begins with knowledge of the field, thus Research begins “at the library” “in a search engine”  Searching  Reviewing  Critically analyzing  Integrating  Effectively summarizing the literature In order to identify and delimit a problem Research – where does it start ?  Specifying and defining testable hypotheses  Designing research to test the hypotheses  Selecting, describing, testing, and treating the participants  Analyzing and reporting the results  Discussing the meaning and implications of the findings as related to the hypotheses Reading - ?  Become familiar with relevant publications.  Read studies of interest.  Read as a practitioner would.  Read the abstract first.  Don’t worry too much about the statistics.  Be critical but objective. How people think research is conducted !  Tenacity  Intuition  Authority  The rationalistic method  The empirical method The Scientific Method  Step 1: developing the problem (defining and delimiting it)  Step 2: formulating the hypotheses  Step 3: gathering the data  Step 4: analyzing and interpreting results Modes of Research Normal science  Previously useful theories  Scientific method Challenges to normal science  Inconsistent findings  Alternative interpretations Paradigm shifts  New theories  New methods Types of Research Analytical  Historical  Philosophic  Reviews  Research synthesis (meta-analysis) Descriptive research  Questionnaire  Interview  Normative survey Types of Research Other types of descriptive research  Case study  Job analysis  Observational research  Developmental studies  Correlational studies Epidemiologic research: public health issues and solutions Experimental research: establishes cause and effect Qualitative research: uses a interpretivist paradigm Research Week 1 – Tutorial Read the journal article prior to attending your scheduled class. NB – 80% attendance is required to complete the class. Course convenor Dr Lindley [email protected]  Week 1 Martin R Lindley INTRODUCITON Welcome to the course Appraising and Applying Evidence for Allied Health Practice PHCM2100 2024  Week 1 Martin R Lindley INTRODUCITON “I would like to show my respects and acknowledge the Bidjigal people who are the Traditional Custodians of the Land on which this campus sits, and to Elders past and present”. PHCM2100 Weekly Schedule  Lectures on-line each week Split into 2x20 min PPT slide decks – with video/audio  Reading task Your tutorial group must read and review the assigned research paper each week ahead of the session.  Tutorial Attend every tutorial and engage in discussion of the assigned research paper  Attendance Attendance will be taken for each tutorial 80% attendance is required for course completion. If you miss a session then you need to contact course convenor and provide the required evidence (eg if you were sick). Please do not be late !  Tutorial leads Thursday 9-10 and 10-11am Martin Lindley Thursday 1-2 2-3 3-4pm Emily Walker Friday 9-10 10-11 11-12pm Elissa Price Friday 1-2 2-3 3-4pm Gynette Reyneke  Assessment Week 4 in-tutorial quiz MCQ format – covering wk1-3 material Week 7 hand in of research paper review – individual Week 10 in-tutorial quiz MCQ format – covering wk4-9 material Week 11 Group presentation of research paper Full details on course moodle page Problems Issue Concerns  If you have any issues you wish to discuss regards the course then please contact the course convenor directly [email protected] Put PHCM2100 in the subject line so that I know its about this course. Read the journal article prior to attending your scheduled class. NB – 80% attendance is required to complete the class. I hope you enjoy the course  Course convenor Dr Lindley [email protected]  Martin R Lindley Week 1 Reflections and learnings PHCM2100 2024 Reading and understanding research papers  4 steps to reading the paper Read read and Review the paper. Take notes. Critique. read again. Approach it from a Record your thoughts Is it strong is it Time spent reading different mindset each about the paper and weak. the paper is always time you read it. What then revisit them next Why do you think well spent. are you looking for ? read through. this ! Read the journal article prior to attending your scheduled class. Engage in the tutorial process. NB – 80% attendance is required to complete the class. Course convenor Dr Lindley [email protected]