Podcast
Questions and Answers
Which of the following is the most accurate definition of 'power' in statistical hypothesis testing?
Which of the following is the most accurate definition of 'power' in statistical hypothesis testing?
- The probability of making a Type I error.
- The probability of correctly rejecting a false null hypothesis. (correct)
- The probability of failing to reject a false null hypothesis.
- The probability of rejecting a true null hypothesis.
How does increasing the alpha level (e.g., from 0.05 to 0.10) typically affect the power of a statistical test, assuming all other factors remain constant?
How does increasing the alpha level (e.g., from 0.05 to 0.10) typically affect the power of a statistical test, assuming all other factors remain constant?
- The effect on power depends on the specific test being used.
- It decreases power because it makes the test more conservative.
- It increases power because it enlarges the rejection region. (correct)
- It has no effect on power; alpha is unrelated to power.
Assuming all other factors are held constant, how does decreasing the population variance generally affect the power of a statistical test?
Assuming all other factors are held constant, how does decreasing the population variance generally affect the power of a statistical test?
- The impact depends on the type of test being conducted.
- Decreasing variance increases power as it sharpens the precision of the sample estimate. (correct)
- Variance has no direct impact on power.
- Decreasing variance decreases power as it reduces the effect size.
In the context of statistical power, what does 'effect size' primarily indicate?
In the context of statistical power, what does 'effect size' primarily indicate?
In hypothesis testing, increasing sample size while holding other factors constant typically leads to:
In hypothesis testing, increasing sample size while holding other factors constant typically leads to:
What is the primary purpose of calculating Cohen's $d$ in statistical analysis?
What is the primary purpose of calculating Cohen's $d$ in statistical analysis?
Which of the following best describes the purpose of a power analysis before conducting a study?
Which of the following best describes the purpose of a power analysis before conducting a study?
In the context of hypothesis testing, a Type II error is:
In the context of hypothesis testing, a Type II error is:
How does the noncentrality parameter ($\delta$) relate to the statistical power of a test?
How does the noncentrality parameter ($\delta$) relate to the statistical power of a test?
In an independent groups t-test, if the sample sizes of both groups increase, the power of the test will typically:
In an independent groups t-test, if the sample sizes of both groups increase, the power of the test will typically:
A Type I error, also known as a false positive, is the incorrect rejection of a ______ null hypothesis.
A Type I error, also known as a false positive, is the incorrect rejection of a ______ null hypothesis.
Decreasing the measurement error in a study generally decreases the statistical power, all else being equal.
Decreasing the measurement error in a study generally decreases the statistical power, all else being equal.
Explain why, in a repeated measures design, controlling individual differences often leads to greater statistical power compared to an independent groups design.
Explain why, in a repeated measures design, controlling individual differences often leads to greater statistical power compared to an independent groups design.
In a scenario where you are comparing two independent means and you have a medium effect size (d=0.5), what is the implication for the required sample size to achieve a desired level of power, compared to a small effect size (d=0.2)?
In a scenario where you are comparing two independent means and you have a medium effect size (d=0.5), what is the implication for the required sample size to achieve a desired level of power, compared to a small effect size (d=0.2)?
According to Cohen's conventions, which of the following Cohen's d values represents a large effect size?
According to Cohen's conventions, which of the following Cohen's d values represents a large effect size?
If a researcher sets the alpha level to 0.01 instead of the typical 0.05, what happens to power, assuming other parameters stay the same?
If a researcher sets the alpha level to 0.01 instead of the typical 0.05, what happens to power, assuming other parameters stay the same?
When calculating power for an independent samples t-test, 'n' in the formula $ \delta = d \sqrt{\frac{n}{2}}$ always refers to the total sample size across both groups.
When calculating power for an independent samples t-test, 'n' in the formula $ \delta = d \sqrt{\frac{n}{2}}$ always refers to the total sample size across both groups.
Briefly explain why 'rounding down' the noncentrality parameter ($\delta$) when consulting a power table is generally recommended.
Briefly explain why 'rounding down' the noncentrality parameter ($\delta$) when consulting a power table is generally recommended.
In cases where you have variance instead of standard deviation, you can calculate standard deviation by taking the ______ root of the variance.
In cases where you have variance instead of standard deviation, you can calculate standard deviation by taking the ______ root of the variance.
Match the following statistical concepts with their corresponding definitions:
Match the following statistical concepts with their corresponding definitions:
How does increasing the effect size influence the required sample size needed to achieve a power of 0.80?
How does increasing the effect size influence the required sample size needed to achieve a power of 0.80?
In a study comparing two independent groups, a researcher finds that increasing the sample size per group from 20 to 80 substantially increases the observed power. What is the primary reason for this increase in power?
In a study comparing two independent groups, a researcher finds that increasing the sample size per group from 20 to 80 substantially increases the observed power. What is the primary reason for this increase in power?
If a statistical test is underpowered, it is more likely to commit a Type I error.
If a statistical test is underpowered, it is more likely to commit a Type I error.
When consulting a power table, if the calculated statistical effect ($\delta$) falls exactly between two values listed in the table, it is conventional to ______ down to the lower value to obtain a conservative estimate of power.
When consulting a power table, if the calculated statistical effect ($\delta$) falls exactly between two values listed in the table, it is conventional to ______ down to the lower value to obtain a conservative estimate of power.
How does using a repeated measures design, when appropriate, impact the required sample size needed to obtain a certain level of statistical power, compared to an independent groups design?
How does using a repeated measures design, when appropriate, impact the required sample size needed to obtain a certain level of statistical power, compared to an independent groups design?
Which of the following is NOT a way to increase power in a statistical test?
Which of the following is NOT a way to increase power in a statistical test?
How does increasing the sample size affect Type II error rate?
How does increasing the sample size affect Type II error rate?
Power calculations are performed using the formula: $Power = 1− \beta$, where $ \beta$ = the probability of being convicted of murder.
Power calculations are performed using the formula: $Power = 1− \beta$, where $ \beta$ = the probability of being convicted of murder.
Cohen defined a small effect size as d = .2. What is the primary problem with small effect sizes in experiments?
Cohen defined a small effect size as d = .2. What is the primary problem with small effect sizes in experiments?
Match the following phrases:
Match the following phrases:
Why should you use the t-test instead of the z-test?
Why should you use the t-test instead of the z-test?
Under the null hypothesis assumed a small effect size; what is assumed under the H1, alternative hypothesis?
Under the null hypothesis assumed a small effect size; what is assumed under the H1, alternative hypothesis?
Sample size has no impact on the power of the statistical test.
Sample size has no impact on the power of the statistical test.
A researcher intends to use an independent samples t-test with samples: 9 old participant and 5 young participants. What is the degrees of freedom?
A researcher intends to use an independent samples t-test with samples: 9 old participant and 5 young participants. What is the degrees of freedom?
Match each design type to its key characteristic:
Match each design type to its key characteristic:
How can you estimate the sample size needed using power?
How can you estimate the sample size needed using power?
You want to measure change over time and need to increase sensitivity with fewer participants. How can you increase this?
You want to measure change over time and need to increase sensitivity with fewer participants. How can you increase this?
It's fine to always check the means when interpreting any differences.
It's fine to always check the means when interpreting any differences.
In hypothesis testing, which of the following statements is true regarding the alternative hypothesis ($H_1$)?
In hypothesis testing, which of the following statements is true regarding the alternative hypothesis ($H_1$)?
Increasing the alpha level (e.g., from 0.05 to 0.10) reduces the probability of making a Type I error.
Increasing the alpha level (e.g., from 0.05 to 0.10) reduces the probability of making a Type I error.
Explain how increasing sample size typically impacts statistical power.
Explain how increasing sample size typically impacts statistical power.
To compute the number of participants needed for a power of 83% in a single-sample t-test, you need to find the ______ using the power table on page 126.
To compute the number of participants needed for a power of 83% in a single-sample t-test, you need to find the ______ using the power table on page 126.
Match each statistical concept with its appropriate definition:
Match each statistical concept with its appropriate definition:
Flashcards
What is Power?
What is Power?
The probability of finding a significant effect if one exists.
Type I error (alpha)
Type I error (alpha)
A "false alarm" or "false positive," the probability of incorrectly rejecting the null hypothesis when it is true. Usually set at .05.
Type II error (beta)
Type II error (beta)
A "miss" or "false negative," The probability of incorrectly accepting the null hypothesis when it is false.
Power (1 - beta)
Power (1 - beta)
Signup and view all the flashcards
Ways to Increase Power
Ways to Increase Power
Signup and view all the flashcards
Effect Size (d)
Effect Size (d)
Signup and view all the flashcards
Statistical effect (δ)
Statistical effect (δ)
Signup and view all the flashcards
Effect Size (d) value determination
Effect Size (d) value determination
Signup and view all the flashcards
Cohen's d values
Cohen's d values
Signup and view all the flashcards
What is Example 1b (p. 35 of the workbook)
What is Example 1b (p. 35 of the workbook)
Signup and view all the flashcards
What is Determine the effect size (d)
What is Determine the effect size (d)
Signup and view all the flashcards
Type II Error
Type II Error
Signup and view all the flashcards
What is 8?
What is 8?
Signup and view all the flashcards
What is N?
What is N?
Signup and view all the flashcards
What is H₀?
What is H₀?
Signup and view all the flashcards
Sample size increase?
Sample size increase?
Signup and view all the flashcards
The likelihood is false...
The likelihood is false...
Signup and view all the flashcards
What are Matched Pairs
What are Matched Pairs
Signup and view all the flashcards
What is lower your...
What is lower your...
Signup and view all the flashcards
Influence power
Influence power
Signup and view all the flashcards
Differences
Differences
Signup and view all the flashcards
Top graph
Top graph
Signup and view all the flashcards
SD is unknown
SD is unknown
Signup and view all the flashcards
Study Notes
- Psychological Research Methodology II, Tutorial 3
Mid-Semester Quiz Details
- Opens at 9am on Monday, March 31st and closes at 5pm on the same day
- The quiz is online and open-book
- Once started, the quiz auto-submits at 5pm
- There is a 90-minute time limit to complete the quiz once it has started
- The exam auto-submits once the timer runs out
- Covers material from the first four lectures and first three tutorials
- The quiz is worth 20% of the final grade
- The format is multiple choice
- Some questions require calculations, while others are conceptual
- There are 14 single-point questions
- There are 4 one and a half-point calculation questions
- Each calculation question has 3 parts, each worth half a mark
Quiz Preparation and Resources
- A practice quiz is available on Blackboard to help prepare
- Tutors offer consult hours to assist with preparation
- Formulae, z-, t-, and power tables from the tutorial workbook should be used for calculations
- Use the formulae taught in this course and avoid using computational formulae learned elsewhere
- Retain 3 decimal places at every step during calculations
- Answers must be your own work (do not collude)
- Questions are randomised between students during the quiz
- Consulting with the lecturer or tutors regarding quiz questions during the quiz is not permitted
- Review lecture slides, tutorial slides, the workbook, and the textbook
- Non-PSYC2010 internet resources should be used with caution
Emergency Contact
- For emergency situations preventing completion of the quiz, submit an application for a deferred quiz with documentation to [email protected]
- A form is available on Blackboard for this purpose
- This email also handles applications for assignment extensions later in the semester
Pooling Variance Example
- Study to investigate the impact of a lead smelter on lead levels in children's blood
- Comparing 10 children near the smelter to 7 children in an unpolluted area
- Children near the smelter show an average lead level of 18.40 (SD = 3.17) mg/100ml
- Children in the unpolluted area show an average lead level of 12.14 (SD = 3.18) mg/100ml
- Formula calculates the pooled variance in a different way: Sp2 = ((N1-1)s12 + (N2-1)s22) / (N1 + N2 - 2)
- Based on previous figures the numbers input accordingly
- Calculating the pooled variance yields a value of 10.074
Further calculations on the same variance
- The standard error of the difference between means can be calculated as 1.565
- Calculate t using the formula: t = (X1 - X2) / sX1-X2
- In this blood level example t = 4.000
- Degrees of freedom are calculated as such: df = 10 + 7-2 = 15
- Making a statistical decision can be done by comparing the absolute calculated t against the t critical value: |tobt(15) = 4.000| > |tcrit(15) = 2.131|
- Reject null hypothesis
- An independent-groups t-test revealed that blood lead levels were significantly higher in kids living closer to the smelter (M = 18.4)
- The levels are higher than the children living in the unpolluted region (M =12.1), t(15) = 4.00, p < .05
Power definition
- Power: Probability of finding a significant effect if one exists
Type I Error
- Type I error (α): A "false alarm" or "false positive."
- Alpha is usually set at .05.
- It represents the probability of incorrectly rejecting Ho when Ho is actually true.
- Example: The probability of finding a significant difference when no reliable difference exists.
- Can only occur when Ho is true.
Type II Error
- Type II error (β): A “miss” or “false negative."
- Probability of incorrectly accepting Ho when it is false.
- Probability of NOT finding a significant difference when a reliable difference does exist.
- Can only occur when Ho is false.
Power Continued
- Power is (1 - β)
- Probability of correctly rejecting a false null hypothesis; power assumes H₁ is true
How Power Influences Decisions
- Decision: Accept Ho, Reality: No difference is a correct choice and a true negative
- Decision: Reject Ho, Reality: No difference is an error and a false positive called type I
- Decision: Accept Ho, Reality: Difference exists is an error, and a false negative called type II
- Decision: Reject Ho, Reality: Difference exists, and a correct rejection
Power Illustration Notes
- The left curve represents innocent people (null)
- The right illustrates the guilty people (alternative)
- The small areas on both ends of the first, innocent, curve are alpha by 2
- This is the chance of wrongly accusing an innocent person, also known as Type I error
- The confusing part is where the two are overlapping, and where you can behave in similar ways
- If a guilty person does this, identifying them as innocent is a type II error
- If a guilty person crosses the decision line correctly you call them guilty; this is power
- As the curves overlap more, it gets harder to tell who is truly guilty or innocent and the more mistakes that can be made
Increasing Power
- Raising alpha from 0.05 to 0.10 lowers the stringency for guilt
- You will only find someone guilty if they act guilty at "10"
- When alpha is 0.05, then they must act guilty at "5"
- Increasing power means being better at correctly identifying guilty people
- One method is to increase the effect size
- You can not control how guilty act to make your work easier
- Another is to reduce variance, meaning making people's behavior more consistent
- You can not always reduce variance always
- Last is to increase the sample size, like more interviewing more people or collecting more evidence
- These three methods cannot always be practically applied in real investigations
Calculating Power Considerations
- Three considerations when calculating power include experimental design, effect size (d), and statistical effect (δ, noncentrality parameter)
- Effect size measures how man standard deviations the distribution of scores are under Ho and H₁
- Estimated based on past research
- Effect size tells us how big the actual difference is in the real world
- The statistical effect tells us how clear that difference is in sample data
- Also called the noncentrality parameter (δ)
- This measures how many standard errors apart the distribution of scores are under Ho and H₁
- With alpha level chosen delta tells how far the true mean Is from the zero in the shifted distribution
- Even if the real difference is substantial, missing It is possible if sample is to small etc
Cohen's d
- Commonly used to describe how big or meaningful an effect is
- If d is around 0.2, the effect is considered small; if it's around 0.5, it's medium; and if it's 0.8 or more, it's large
- However, these are not hard rules, its more of a guideline
Single Sample t-test for Finding Power
- Determine the effect size (d), which is independent of sample size: d = (μ1 - μ0)/σ
- Calculate the statistical effect (δ), which is dependent on sample size: δ = d√N
- For a single sample t-test, N = n
- Look up the power in the table (p. 126) for the calculated δ and given α
Single Sample t-test to Find Sample Size:
- Determine the effect size (d)
- Use the table (p. 126) to look up the statistical effect required to yield a particular power for the given alpha
- Calculate the required sample size: N = (δ/d)^2
Single Sample t-test Example
- Example 1a comes from page 35 of the workbook
- Data from tests on young adult memory suggest that, on average, they can remember 11.50 objects out of 20, with a variance of 19.03
- Collected from random samples of 22 older adults, it showed that older adults were able to remember an average of, 13.25
- Calculate the power of the test in order to replicate study with older adults can be replicated
- Null hypothesis: Older adults' memory is the same as the young; Alternative hypothesis: older is actually 13.25
- μ0 = 11.50, μ1 = 13.25, σ2 = 19.03, N = 22
- First determine effect size: σ2 = √19.030 = 4.362, d = μ1-μ0/ σ 13.250 - 11.500/4.362= 0.40
- Second calculate what statistical effect is: δ= d√N= .400√22 = .401 x 4.689 = 1.881
- Then look up the power table on (p. 126)
- Therefore, round “down” to be a conservative i.e 1.80, power = 44%
Sample t-Test Interpretation
- The single t-test has given results of the memory test showing that if true mean of other adulate 13.3, then there is a 44% chance correctly rejecting this, with a sample of 22
- If H0 is false, power is undefined Power calculations make sense when hOh0 is false
- Defining power requires the formula form and expressing percentage
Finding Sample t-Test to Find Sample Participants
- Determining the participants needed to achieve the specific power is done with this formula: N =(δ/d)^2
- Determine d effect size we know
- Finding delta is done with the power table and alpha level of given.
- This finds what statistical effect is required for the desired power
Independent Groups t-test Notes
- Finding power from a given sample size involves a few steps
- Determine size effect (d): μ-μ0/σ
- Calculate statistical effect: 8 = d n/ 2, n = sample of each group Look up in power table (p. 126) the result in given aplha
Independent Groups t-test continued
- The independent test can be used for a specific power, or the sample can be used from particular power
- μ − μ0
- Determine effect size dependent of sample size d
- Find the table to locate the required to yield specific power of the given α test
- calculate this using calculated required sample site
Independent Groups T-test Example
- Test of whether people use more positive words then negative
- medium effect of .05, a new independent test needed with 40 people and a
- What is power of this result ( = .05, with n for each group is 40 )
Effect size determination
- previous research would be ""medium"" effect, = ,50. sample means populations of score which has 1/2
- Statistical Effect
- the formulas are:. d equal equal to d: n1 = 4 , 2 = 40 =N
- d 2 == .,500.
Final Independent Calculation
- A give of is 0 5, a a statical effect from the result ,(0 has the power of or rounded down by 2.2
- note: we don't h t have 2: but we he hve s so use at
- make sure that your you is, and the d n1 n and If false, and the word, with ample which 4 group
- use is look give has so test word,
- So a give for, this if give
- To
Important multiple choice points:
- power equals finding of a. of
- Alapha and of we call
Revision Questions Tutorial 1 and 2:
- Identify the independent and dependent variables, experimental design, and the appropriate statistical test.
- Example 1: Studying the effects of a new memory-enhancing drug on Alzheimer's patients by testing before and after administration.
- IV: Drug administration or time (pre/post)
- DV: Memory test scores
- Design: Repeated measures
- Test: Repeated measures t-test
- Example 2: Finding out if the mean IQ of statisticians differs from the general population (mean IQ 100, standard deviation 15) by giving an IQ test to 50 statisticians.
- IV: Job type (statistician vs. population)
- DV: IQ
- Design: Single sample
- Test: Single sample z-test
- Example 3: Studying the effects of alcohol on motor coordination by comparing a group given a moderate dose of alcohol to another group given no alcohol.
- IV: Alcohol dosage (moderate vs. none)
- DV: Motor coordination
- Design: Independent groups
- Test: Independent groups t-test
- Example 4: Examining if males and females are persuaded differently by a female speaker, with each male participant matched to a female participant of the same age, IQ, and occupation.
- IV: Participant gender (males vs. female)
- DV: Speaker persuasion
- Design: Matched samples
- Test: Repeated measures t-test
- Example 5: A student comparing the number of driving lessons his friends took to the average number of lessons reported by a driving school (23 lessons).
- IV: Friends vs. population
- DV: Number of lessons
- Design: Single sample
- Test: Single sample t-test
Mid-Semester Quiz: Scope and Resources
- All content from first 4 lectures
- All content from first 3 tutorials
- Introductory statistics will be useful
- Include lecture slides, tutorials videos, workbooks, and textbooks
- Use non-PSYC2010 resources carefully
Key Concepts to Review
- Normal distributions: Shape, center, and spread
- z-scores and z-transformations: Definition, calculation, and interpretation
- z-tables: How to find probabilities/percentiles
- Using z-scores for significance testing: Single scores and single samples
- Sampling error and sampling bias: Definition and impact
- Inferential versus descriptive statistics: Difference and application
- Sampling distribution of means: Definition and usefulness
- Hypothesis testing: Types of hypotheses, formulation, and null hypothesis testing
- Choosing between z and t-tests: Criteria and assumptions for selection
t-Test Types and How to Conduct Them
- Single sample: Comparing a sample mean to a known population mean
- Independent groups: Comparing means of two unrelated groups
- Repeated measures: Comparing means from the same subjects under different conditions
- Select the correct test
- Review your formulas
- Consider the degrees of freedom
- Review the means when interpreting the results
- Ensure they meet the basic assumptions regarding the normality of data and equality of variance
More Concepts to Review
- Advantages and disadvantages of independent vs. repeated measures designs/tests
- Power and its relationship to key variables (d, N, δ)
- How power and effect sizes are used by researchers
- Types of errors (Type I, Type II) and associated probabilities
- Factors that influence power: explanation and reasons
- Common rules from Cohen (effect sizes)
- Method for calculating power for different t-tests
- Consider the N needed for a given power result
Additional Notes
- Effect Size (Cohen's d): It's the measure of the distance between two means
- A small effect size: d≈ 0.2 results in an overlapping of groups
- If variance given: Standard Deviation variance equals the square root result
- Delta Combines effect size plus sample size, the detectable outcome from that sampling result
- Independent samples, Where n per group and independent groups created
- Under null hypothesis, with distribution curves overlapping the effect size from the h1 curve moves from HO
- Power is under h, but only if this result rejects H0, also known as a false result
Other Reminders
- It is best practice to down the delta table in order to be a more conservative
- Conservative helps in avoiding overestimation
- Ex delta equals two point to for look up the two point.
Studying That Suits You
Use AI to generate personalized quizzes and flashcards to suit your learning preferences.
