Emmeans, pairs, contrasts, and all different p-values

101 views Asked by At

The main regression formula I am using is as follows (Both the Group and Time variables are binary):

lmer.fit <- lmerTest::lmer(corrmcc ~ Group * Time + (1|ID), 
                           data = sm.dat)
summary(lmer.fit)

# Fixed effects:
#                   Estimate Std. Error     df t value Pr(>|t|)    
# (Intercept)         429.92      36.14  65.74  11.896   <2e-16 ***
# GroupTTNS           -37.75      51.64  65.74  -0.731    0.467    
# TimeEoS             -55.68      34.92  40.36  -1.595    0.119    
# GroupTTNS:TimeEoS   -23.23      50.47  40.73  -0.460    0.648    

As you can see, the Time variable (or TimeEoS) has a p-value of > 0.05. That is, there is no significant difference in the outcome between the two time points.

Then I applied the emmeans function with a bar (|) for a post-hoc analysis as follows:

pairs(emmeans(lmer.fit, ~ Time | Group)) 
Group = Control:
 contrast       estimate   SE   df t.ratio p.value
 Baseline - EoS     55.7 35.0 42.0   1.592  0.1189

Group = TTNS:
 contrast       estimate   SE   df t.ratio p.value
 Baseline - EoS     78.9 36.5 42.7   2.161  0.0364

Here, we can observe that for the TTNS group, the difference between the baseline and EoS is statistically significant (i.e., p-value = 0.0364, < 0.05).

However, when I applied the same function but with an asterisk (*), I obtained an opposite result as follows:

pairs(emmeans(lmer.fit, ~ Time * Group))
 contrast                         estimate   SE   df t.ratio p.value
 **Baseline Control - EoS Control       55.7 35.0 42.0   1.592  0.3940**
 Baseline Control - Baseline TTNS     37.8 51.6 66.9   0.731  0.8843
 Baseline Control - EoS TTNS         116.7 53.4 71.2   2.184  0.1376
 EoS Control - Baseline TTNS         -17.9 52.8 69.8  -0.340  0.9864
 EoS Control - EoS TTNS               61.0 54.6 73.6   1.118  0.6797
 **Baseline TTNS - EoS TTNS             78.9 36.5 42.7   2.161  0.1508**

Here, the difference between the baseline and EoS in the TTNS group is not significant with a p-value of 0.15.

I am sure that I am missing something, but I am having trouble finding what is causing the differences. Any suggestions and comments would be greatly appreciated!

0

There are 0 answers