Power and Effective size

1. Overview

2. Description

2.1 Power ($1-\beta$)

The probability of correctly rejecting a false null hypothesis

$$Power=P(reject\: H_{0}|H_{1}\: is\: true)=1-\underbrace{P(not\: rejecting\: H_{0}|H_{0}\: false)}_{Type\: 2\: error}\\=1-\beta=P(not\: making\: Type\: 2\: error)$$

Particular	Interpretation
Sample Size	Larger sample more power
Effect Size	Larger Effect sieze more power
Alpha Level	Higher Alphas more power
Directionality	One tail more power

2.1.1 Power Analysis

• Non-centrality parameter

$$\delta =\frac{d}{\sigma \sqrt{\frac{1}{n_{1}}+\frac{1}{n_{2}}}}$$

$\sigma$ is often assumed to be 1 because assumed Variances are same in 2 groups and N's are same in 2 groups.

When $n_{1}=n_{2}$

$$\delta =d\sqrt{\frac{n_{k}}{2}}\\
n_{k}=2(\frac{\delta }{d})^{2}$$

When $n_{1}\neq n_{2}$

$$\bar{n}_{h}=\frac{2}{\frac{1}{n_{1}}+\frac{1}{n_{2}}}=\frac{2n_{1}n_{2}}{n_{1}+n_{2}}\\
\delta =d\sqrt{\frac{\bar{n}_{h}}{2}}$$

2.1.2 Problems with low power

• False Negatives
• Inflated effect size estimates
• Lower positive predictive value

 

2.2 Effect Size

The effect size is the degree of distance between the null hypothesis and alternative hypothesis distributions.

2.2.1 Types

• Cohen's d

Cohen's d is defined as the difference between two means divided by a standard deviation for the data, i.e.

$$d=\frac{\bar{x}_{1}-\bar{x}_{2}}{s}=\frac{\mu_{1}-\mu_{2}}{s}=t\sqrt{\frac{n_{1}+n_{2}}{n_{1}n_{2}}}$$

Jacob Cohen defined s, the pooled standard deviation, as (for two independent samples):

$$s=\sqrt{\frac{(n_{1}-1)s_{1}^{2}+(n_{2}-1)s_{2}^{2}}{n_{1}+n_{2}-2}}$$

Cohen's d	Interpretation
0.2	Small
0.5	Mederate
0.8	Large

3. Reference

https://en.wikipedia.org/wiki/Power_(statistics)

https://en.wikipedia.org/wiki/Effect_size

http://www.psychology.emory.edu/clinical/bliwise/Tutorials/SPOWER/spowesize.htm

http://tysonbarrett.com/EDUC-6600/Slides/u02_Ch8_power.pdf

https://www.google.com/search?q=power+and+effect+size&sxsrf=ACYBGNTZ8cBgg3XbiXfPuAlP9EBfOEvL6w:1580895239471&source=lnms&tbm=isch&sa=X&ved=2ahUKEwjmyLPZjbrnAhUVE4gKHVkLBKoQ_AUoAXoECA0QAw&biw=1920&bih=937#imgrc=HcaO42kGxBw_PM

https://www.youtube.com/watch?v=6_Cuz0QqRWc

https://www.youtube.com/watch?v=6uYNVCy-8NA

https://www.youtube.com/watch?v=tTgouKMz-eI

https://www.youtube.com/watch?v=STO1NtVR2HI

https://effectsizefaq.com/2010/05/31/what-is-statistical-power/

https://statisticsbyjim.com/glossary/effect/

https://osf.io/sf9cv/