Jeromy Anglim's Blog: Psychology and Statistics

Friday, November 7, 2008

Using R to test for significant differences between two nonindependent correlations

I was recently asked about a quick and easy way to see whether the correlation between IV1 and DV is statistically significantly different from IV2 and DV. The follow post set out how this analysis can be conducted in R. The instructions assume that you have never used R before and are running Windows.

1) Download and install R for windows

2) Install the psych package
Run the following line of code in the command prompt
install.packages("psych")

3) Load psych package
Run this code
library(psych)

4) Run the test
Enter code like this:

paired.r(.2,.3,.93,100)

This above code tests whether correlation of 0.20 is statistically different from 0.30 given that the two predictor variables are correlated 0.93, assuming a sample size of 100.

The output in R looks like this:
paired.r(.2,.3,.93,100)
$test
[1] "test of difference between two correlated correlations"
$t
[1] -2.832009
$p
[1] 0.005614689

It could be written up as:
The correlation between IV1 and DV (r = 0.30) was significantly larger than the correlation between IV2 and DV (r = 0.20), n=100, p=0.006.
By Jeromy Anglim on Friday, November 07, 2008 Labels: ,

12 comments:

  1. Richard Moulding:
    "point[s] out for the non-r users there is a macro for SPSS correlations Meng test by anthony hayes at
    http://psyphz.psych.wisc.edu/~shackman/meng.sps
    "
    http://sites.google.com/site/richardmouldinghomepage/

    ReplyDelete

  2. How do you determine the 0.93 (the two predictor variables)

    ReplyDelete
    Replies

    1. If you have three variables (X, Y, Z), then there are three possible correlations X with Y, X with Z, and Z with Y.

      If you have raw data, just run a correlation matrix on the three variables.

      If you are interpreting an existing correlation matrix, just look up the three relevant correlations.

      Delete

  3. What is the basis for this calculation? Or how do I cite the procedure in a paper? Perhaps more importantly--I may need to explain (or at least support) it during my defense : ) My limited understanding is that with independent samples a Fisher Z test is used, but with dependent samples there is some controversy about how to do this.

    ReplyDelete

  4. Howell "Statistical Methods for Psychology" provides further explanation with references.

    ReplyDelete
    Replies

    1. Thanks for providing the R code for testing the difference between multiple correlations.

      Is this test suitable for Spearman Rank (rs) correlations?

      Zar (1996) advised that hypothesis tests can be carried out on rs correlations, but suggests that the the variance of z be calculated as sqrt(1.06/n-3), rather than sqrt(1/n-3).

      Many thanks!

      Delete

  5. Jeromy,

    Does this calculation work for partial correlations as well? I have 3 variables and have run partial correlations on the three possible relationships between the 3 variables. I want to assess if the partial correlation between variables 1 and 2 controlling for 3 is significantly larger than the partial correlation between variables 1 and 3 controlling for 2.

    Thanks

    ReplyDelete

  6. Hello, this is very interesting as I am currently trying to calculate to see if one correlation is greater than another when looking at the same sample (so nonindependent correlations). What exactly are these computed values in the example : [1] -2.832009[1] 0.005614689
    Must they be used after or is it that the first being of a lesser value than the other means that the first correlation entered is indeed lesser than the other? And where does the P in the example come from? Thanks

    ReplyDelete
    Replies

    1. t represent a t-statistic (from Hotelling's test (dependent correlations) , and the p is in this case the two-tailed p-value associated with the t-statistic. Thus, in a typical context if p < .05, you might conclude that there is a statistically significant difference between the correlations.

      Delete

  7. Hi,

    I posted this question a little while ago and wonder if anyone can offer some guidance for me?

    Particularly, I'd like to know if this test is suitable for Spearman Rank (rs) correlations?

    Zar (1996) advised that hypothesis tests can be carried out on rs correlations, but suggests that the the variance of z be calculated as sqrt(1.06/n-3), rather than sqrt(1/n-3).

    Thanks!

    ReplyDelete

  8. Hi Jeromy,

    Thanks for the post. I was wondering if you could answer a related question. I would like to test the difference between two correlations, but the structure of my data is different than what you describe here.

    My subjects were measured for two behavioral variables. I measured subjects twice, once after experimental treatment and once after control treatment (order was balanced). I would like to see if the correlation between the two behavioral variables in the control treatment differs from the correlation in the experimental treatment. From my understanding, Fisher's Z test is inappropriate since my behavioral variables are not normally distributed and the repeated measures create non-independence. I've searched quite a bit for how to analyze the data this way, but no luck. Do you have suggestions as to what the proper analysis is and how to perform it in R?

    Thanks

    ReplyDelete

  9. I believe that Olkin and Finn provide a formula. See particularly equation 5. I'm not sure whether an R implementation exists.

    Olkin, I. & Finn, J.D. (1995). Correlations redux. Psychological Bulletin, 118, 155.

    ReplyDelete

Subscribe to: Post Comments (Atom)