Jeromy Anglim's Blog: Psychology and Statistics


Wednesday, May 5, 2010

Statistical Power Analysis in G*Power 3

G*Power 3 is an excellent piece of software for performing statistical power analysis. It is particularly useful for applied researchers who need to perform a power analysis as part of their research. The software is free, runs on Windows, and provides a user friendly GUI. G*Power 3 can be downloaded here This post discusses the features of G*Power 3 and provides examples of some of the useful plots that can be generated.

Overview

Statistical power is the probability of correctly rejecting the null hypothesis when it is false in a given sample. For further discussion of Power Analysis, see Jacob Cohen's classic 1992 Psychological Bulletin Power Primer or Statsoft, or Faul, Erdfelder, Lang, and Buchner, 2007.

Power analysis is particularly useful when planning a study. There are many trade-offs when designing a study. Awareness of the trade-offs and how they relate to statistical power is particularly important. For example, you might be confronted with the decision of whether to use a subtle experimental manipulation (small effect size) that is of greater applied relevance or a stronger experimental manipulation (large effect size) which is less realistic. Being aware of the relative power of the two options helps to highlight whether the less realistic, but more statistically powerful, option is the better choice.

The website for G-Power 3 contains help material for many common scenarios. G*Power 3 makes it easy to get precise values of your statistical power for your given scenario. If G*Power 3 is insufficient for your needs, you may want to consider writing a simulation in a language such as R (see William Revelle's comments)

The following four plots show examples of the great plots that G*Power 3 can generate. I've also selected these four plots because I think that researchers in psychology should have an intuition about the statistical power of what may be the most common statistical tests: (a) the t-test of differences between independent groups, and (b) the correlation coefficient. Most studies in psychology have many variables and many hypotheses. However, many, but not all, of these hypotheses can be reduced to correlations between two numeric variables (e.g., IQ and job performance) or differences in a numeric variable between two groups (happiness levels between those receiving counselling and those not receiving counselling). Thus, internalising the approximate values of these four plots should help you when you design studies and when you read and evaluate the research literature.

What statistical power do I have given my sample size and assumed population correlation?

The rough rules of thumb can help: r = .1 is small; r = .3 is medium; r = .5 is large.

What statistical power do I have given my sample size and assumed population difference between group means?

Rough rules of thumb: d = .2 is small; r = .5 is medium; r = .8 is large.

What sample size do I need given a desired power an assumed population correlation?

The following two plots are useful when deciding what sample you need.

What sample size do I need given a desired power an assumed population difference between group means?


Related Resources