We discussed how to conduct a 2-way factorial ANOVA in this tutorial, and we talked about the three different types of Sum of Squares here. We will build on these and discuss how to run post hoc analyses when you have a significant interaction. We will use the `Moore`

dataset from the `carData`

package in R. This data frame consists of subjects in a “social-psychological experiment who were faced with manipulated disagreement from a partner of either of low or high status. The subjects could either conform to the partner’s judgment or stick with their own judgment.” (John Fox, Sanford Weisberg and Brad Price (2018). carData: Companion to Applied Regression Data Sets. R package version 3.0-2. https://CRAN.R-project.org/package=carData)

The variables we will use in our analyses are:

`partner.status`

: Partner’s status. A factor with levels*high*and*low*.`fcategory`

: F-Scale Categorized. A factor with levels*low*,*medium*, and*high*.`conformity`

(outcome): Number of conforming responses in 40 critical trials.

First, we visualize the data. The following boxplot shows the distribution of scores on the conformity variable within each combination of `partner.status`

and `fcategory`

.

We can also get a sense of whether an interaction is present by looking at an interaction plot.

Recall from the **two-way ANOVA** tutorial that if there is an interaction, the difference in means between treatment levels will be *different* depending on the level of the other factor, i.e. the plots of means will *not* be parallel. Looking at this plot, there appears to be an interaction between `partner.status`

and `fcategory`

. The difference in means in the two partner status levels is small when F-score category is low but larger when the F-score category is medium or high.

Next, we will run our 2-way ANOVA, and get the following results (Note that we are using type III Sum of Squares):

Source | Type III Sum of Squares | df | Mean Square | F | Sig. |
---|---|---|---|---|---|

partner.status | 239.562 | 1 | 239.562 | 11.425 | .002 |

fcategory | 36.019 | 2 | 18.009 | .859 | .431 |

partner.status*fcategory | 175.489 | 2 | 87.744 | 4.185 | .023 |

Error | 817.764 | 39 | 20.968 |

From these results, we can conclude that, based on a significance level of \(\alpha = 0.05\):

- The main effect of Partner Status (\(p = 0.002\)) is significant
- The main effect of F-Score Category (\(p = 0.431\)) is not significant
- The interaction effect (\(p = 0.023\)) is significant

The interaction effect is significant in the overall ANOVA, but that knowledge is not meaningful unless you look at the pairwise comparisons. Within each level of `fcategory`

(“low”, “medium”, and “high”) we will perform pairwise comparisons to `partner.status`

.

Partner Status | \(df_1\) | \(df_2\) | F-ratio | p-value |
---|---|---|---|---|

Low | 2 | 39 | 2.323 | 0.111 |

High | 2 | 39 | 2.138 | 0.132 |

The effect of F-Score appears to not be significant in either case. That is, the difference in means between F-score categories are not significantly different from each when partner status is low, nor are they significantly different when partner status is high.

Next, we will repeat this for partner status in different levels of F-score category:

F-Score Category | \(df_1\) | \(df_2\) | F-ratio | p-value |
---|---|---|---|---|

Low | 1 | 39 | 11.486 | 0.002 |

Medium | 1 | 39 | 6.899 | 0.012 |

High | 1 | 39 | 0.105 | 0.748 |

At an \(\alpha = 0.05\) level, the effect of partner status within the low and medium F-score categories are significant (\(p = 0.002\) and \(p = 0.012\) respectively). Partner status does not appear to have a significant effect on conformity for subjects with high F-scores.

Finally, we will do pairwise comparisons for levels of partner status within the F-score categories that yielded significant results. That is, we’ll compare high versus low status among subjects with low F-scores, and then we’ll do the same among subjects with medium F-scores. We ignore the high F-score subjects because we did not find a significant effect. Note that, because partner status only has two categories, we only perform a single pairwise comparison within each F-score category.

F-Score Category | Contrast | Estimate | SE | \(df\) | \(t\)-ratio | p-value |
---|---|---|---|---|---|---|

Low | Low - High | -8.500 | 2.508 | 39 | -3.389 | 0.002 |

Medium | Low - High | -7.023 | 2.674 | 39 | -2.627 | 0.012 |