07-anova.Rmd

# Moderation with Analysis of Variance (ANOVA) {#anova}

> Key concepts: eta-squared, between-groups variance, within-groups variance, *F* test on analysis of variance model, pairwise comparisons, post-hoc tests, one-way analysis of variance, two-way analysis of variance, balanced design, main effects, moderation, interaction effect.

Watch this micro lecture on moderation with analysis of variance for an overview of the chapter.

```{r, echo=FALSE, out.width="640px", fig.pos='H', fig.align='center', dev="png", screenshot.opts = list(delay = 5)}
knitr::include_url("https://www.youtube.com/embed/klV2FFgH9OU", height = "360px")
```

### Summary {.unnumbered}

```{block2, type='rmdimportant'}
How do we test mean differences for three or more groups and what if group effects are not the same for all participants?
```

Imagine an experiment in which participants watch a video promoting a charity. They see George Clooney, Angelina Jolie, or no celebrity endorse the charity's fund-raiser. Afterwards, their willingness to donate to the charity is measured. Which campaign works best, that is, produces highest average willingness to donate? Or does one campaign work better for females, another for males?

In this example, we want to compare the outcome scores (average willingness to donate) across more than two groups (participants who saw Clooney, Jolie, or no celebrity). To this end, we use analysis of variance. The null hypothesis tested in analysis of variance states that all groups have the same average outcome score in the population.

This null hypothesis is similar to the one we test in an independent-samples *t* test for two groups. With three or more groups, we must use the variance of the group means (between-groups variance) to test the null hypothesis. If the between-groups variance is zero, all group means are equal.

In addition to between-groups variance, we have to take into account the variance of outcome scores within groups (within-groups variance). Within-groups variance is related to the fact that we may obtain different group means even if we draw random samples from populations with the same means. The ratio of between-groups variance over within-groups variance gives us the *F* test statistic, which has an *F* distribution.

Differences in average outcome scores for groups on one independent variable (usually called *factor* in analysis of variance) are called a main effect. A main effect represents an overall or average effect of a factor. If we have only one factor in our model, for instance, the endorser of the fund-raiser, we apply a one-way analysis of variance. With two factors, we have a two-way analysis of variance, and so on.

With two or more factors, we can have interaction effects in addition to main effects. An interaction effect is the joint effect of two or more factors on the dependent variable. An interaction effect is best understood as different effects of one factor across different groups on another factor. For example, Clooney may increase willingness to donate among females but Jolie works best for males.

The phenomenon that a variable can have different effects for different groups on another variable is called moderation. We usually think of one factor as the predictor (or independent variable) and the other factor as the moderator. The moderator (e.g., sex) changes the effect of the predictor (e.g., celebrity endorser) on the dependent variable (e.g., willingness to donate).

#### Essential Analytics {.unnumbered}

In SPSS, we use the *One-Way ANOVA* option in the *Compare Means* submenu for one-way analysis of variance and the *Univariate* option in the *General Linear Model* submenu for two-way analysis of variance.

```{r ANOVAtable, echo=FALSE, out.width="60%", fig.pos='H', fig.align='center', fig.cap="SPSS table of main and interaction effects in a two-way analysis of variance."}
knitr::include_graphics("figures/S7_AE1.png")
```

The significance tests on the main effects and interaction effect are reported in the **Tests of Between-Subjects Effects** table. Figure \@ref(fig:ANOVAtable) offers an example. The tests on the main effects are in the red box and the green box contains the test on the interaction effect. The APA-style summary of the main effect of endorser is: *F* (2, 137) = 8.43, *p* \< .001, eta^2^ = .10. Note the two degrees of freedom in between the brackets, which are marked by a blue ellipse in the figure. You get the effect size eta^2^ by dividing the sum of squares of an effect by the corrected total sum of squares (in purple ellipses in the figure): 37.456 / 389.566 = 0.10.

Interpret the effects by comparing mean scores on the dependent variable among groups:

1. If there are two groups on a factor, for example, females and males, compare the two group means: Which group scores higher? For example, females score on average 5.05 on willingness to donate whereas the average willingness is only 4.19 for males. The _F_ test shows whether or not the difference between the two groups is statistically significant.

2. If a factor has more than two groups, for example, Jolie, Clooney, and no celebrity endorser, use post-hoc comparisons with Bonferroni correction. The results tell you which group scores on average higher than another group and whether the difference is statistically significant if we correct for capitalization on chance.

3. If you want to interpret an interaction effect, create means plots such as Figure \@ref(fig:ANOVAplots). Compare the differences between means across groups. In the left panel, for example, we see that the effect of sex on willingness to donate (the difference between the mean score of females and the mean score of males) is larger for Clooney (pink box in the middle) than for no celebrity endorser (pink box on the left), and it is smallest for Angelina Jolie (pink box on the right). Similarly, we see that the effect of seeing Clooney instead of no celebrity endorser is larger for females (right-hand panel, pink box on the right) than for males (right-hand panel, pink box on the left).

```{r ANOVAplots, echo=FALSE, out.width="100%", fig.pos='H', fig.align='center', fig.cap="SPSS means plots of the interaction effect of sex and endorser on willingness to donate. Note that the pink boxes have been added manually to aid the interpretation."}
knitr::include_graphics("figures/S7_AE2.png")
```

## Different Means for Three or More Groups

Celebrity endorsement theory states that celebrities who publicly state that they favour a product, candidate, or cause, help to persuade consumers to adopt or support the product, candidate, or cause [for a review, see @RefWorks:3940; for an alternative approach, see @RefWorks:3941].

Imagine that we want to test if the celebrity who endorses a fund raiser in a fund-raising campaign makes a difference to people's willingness to donate. We will be using the celebrities George Clooney and Angelina Jolie, and we will compare campaigns with one of them to a campaign without celebrity endorsement.

```{r clooneyjolie, echo=FALSE, out.width="50%", fig.pos='H', fig.align='center', fig.cap="George Clooney and Angelina Jolie. Photo Clooney by Angela George [CC BY-SA 3.0](https://upload.wikimedia.org/wikipedia/commons/3/32/GeorgeClooneyHWoFJan12.jpg). Photo Jolie by Foreign and Commonwealth Office [CC BY 2.0](https://upload.wikimedia.org/wikipedia/commons/a/ad/Angelina_Jolie_2_June_2014_%28cropped%29.jpg), via Wikimedia Commons."}
# Include portraits: Clooney, Jolie.
knitr::include_graphics("figures/ClooneyJolie.png")
```

Let us design an experiment to investigate the effects of celebrity endorsement. We sample a number of people (participants), whom we assign randomly to one of three groups. We show a campaign video with George Clooney to one group, a video with Angelina Jolie to another group, and the third group---the control group---sees a campaign video without celebrity endorsement. So we have three experimental conditions (Clooney, Jolie, no endorser) as our independent variable.

Our dependent variable is a numeric scale assessing the participant's willingness to donate to the fund raiser on a scale from 1 ("absolutely certain that I will not donate") to 10 ("absolutely certain that I will donate"). We will compare the average outcome scores among groups. If groups with Clooney or Jolie as endorser have systematically higher average willingness to donate than the group without celebrity endorsement, we conclude that celebrity endorsement has a positive effect.

In statistical terminology, we have a categorical independent (or predictor) variable and a numerical dependent variable. In experiments, we usually have a very limited set of treatment levels, so our independent variable is categorical. For nuanced results, we usually want to have a numeric dependent variable. Analysis of variance was developed for this kind of data [@RefWorks:3955], so it is widely used in the context of experiments.

### Mean differences as effects {#anova-meandiffs}

Figure \@ref(fig:anova-means) shows the willingness to donate scores for twelve participants in our experiment. Four participants saw Clooney, four saw Jolie, and four did not see a celebrity endorser in the video that they watched.

```{r anova-means, fig.pos='H', fig.align='center', fig.cap="How do group means relate to effect size?", echo=FALSE, screenshot.opts = list(delay = 5), dev="png", out.width="540px"}
# Goal: Illustrate that differences between group means represent effects (effect size given by eta^2).
# Generate 4 randomobservations with mean 6.4 (Clooney), 4 observations with mean 6.8 (Jolie), and 4 observations with mean 3.3 (no endorser). Use colour for the treatment factor (3 levels). Represent observations in a dotplot, each with a separate value on the x axis, clustered by factor level (experimental condition). Display group means as horizontal line segments (coloured by factor level). Add vertical double-sided arrows between each pair of group means to illustrate group differences. Display eta^2 for the data. Allow user to change the group means and update the plot, mean (difference) lines, and eta^2.
knitr::include_app("http://82.196.4.233:3838/apps/anova-means/", height="490px")
```

<A name="question7.1.1"></A>

```{block2, type='rmdquestion'}
1. In the sample of (12) participants displayed as dots in Figure \@ref(fig:anova-means), what do the double-sided vertical arrows represent? [<img src="icons/2answer.png" width=115px align="right">](#answer7.1.1)
```

<A name="question7.1.2"></A>

```{block2, type='rmdquestion'}
2. According to this figure, does a celebrity endorser matter to the willingness to donate? Explain your answer. [<img src="icons/2answer.png" width=115px align="right">](#answer7.1.2)
```

<A name="question7.1.3"></A>

```{block2, type='rmdquestion'}
3. How do the double-sided vertical arrows relate to effect size (eta^2^)? Change the group means (and update the graph) and explain what you see. [<img src="icons/2answer.png" width=115px align="right">](#answer7.1.3)
```

A group's average score on the dependent variable represents the group's score level. The group averages in Figure \@ref(fig:anova-means) tell us for which celebrity the average willingness to donate is higher and for which situation it is lower.

Random assignment of test participants to experimental groups (e.g., which video is shown) creates groups that are in principle equal on all imaginable characteristics except the experimental treatment(s) administered by the researcher. Participants who see Clooney should have more or less the same average age, knowledge, and so on as participants who see Jolie or no celebrity. After all, each experimental group is just a random sample of participants.

If random assignment was done successfully, differences between group means can only be caused by the experimental treatment (we will discuss this in more detail in Chapter \@ref(confounder)). Mean differences are said to represent the *effect* of experimental treatment in analysis of variance.

Analysis of variance was developed for the analysis of randomized experiments, where effects can be interpreted as causal effects. Note, however, that analysis of variance can also be applied to non-experimental data. Although mean differences are still called effects in the latter type of analysis, these do not have to be causal effects.

In analysis of variance, then, we are simply interested in differences between group means. The conclusion for a sample is easy: Which groups have higher average score on the dependent variable and for which are they lower? A means plot, such as Figure \@ref(fig:anova-meansplot), aids interpretation and helps communicating results to the reader. On average, participants who saw Clooney or Jolie have higher willingness to donate than participants who did not see a celebrity endorser.

```{r anova-meansplot, echo=FALSE, fig.pos='H', fig.align='center', fig.cap="A means plot showing that average willingness to donate is higher with a celebrity endorser than without a celebrity endorser. As a reading instruction, effects of endorsers are represented by arrows.", fig.asp=0.6, out.width="70%"}
# Insert means plot for celebrity endorsement example.
d <- data.frame(endorser = factor(c("Nobody","Clooney","Jolie"), levels = c("Nobody","Clooney","Jolie")), willingness_av = c(3, 6, 7), const = 1)
library(ggplot2)
ggplot(d, aes(endorser, willingness_av)) + 
  geom_point(size = 3, color=brewercolors["Blue"]) + 
  geom_line(aes(group = const), size = 1, color=brewercolors["Blue"]) + 
  geom_segment(aes(x = 1, xend = 3, y = d[1,2], yend = d[1,2]),
               linetype = "dashed", color = "black") +
  geom_segment(aes(x = 2, xend = 2, y = d[1,2], yend = (d[2,2] - 0.1)),
               color = "darkgrey",
               arrow = arrow(length = unit(2,"mm"),
                                 # ends = "both",
                                 type = "closed")) +
  geom_text(aes(x = 2.02, y = (d[1,2] + d[2,2])/2,
            label = "Clooney effect",
            hjust = 0), color = "darkgrey"
            ) +
  geom_segment(aes(x = 3, xend = 3, y = d[1,2], yend = (d[3,2] - 0.1)),
               color = "darkgrey",
               arrow = arrow(length = unit(2,"mm"),
                                 # ends = "both",
                                 type = "closed")) +
  geom_text(aes(x = 2.98, y = (d[1,2] + d[3,2])/2,
            label = "Jolie effect",
            hjust = 1), color = "darkgrey"
            ) +
  theme_general() + 
  scale_y_continuous(limits = c(1, 10), breaks = c(1, 5, 10)) + labs(x = "Endorser", y = "Average willingness to donate")
rm(d)
```

Effect size in an analysis of variance refers to the overall differences between group means. We use eta^2^ as effect size, which gives the proportion of variance in the dependent variable (willingness to donate) explained or predicted by the group variable (experimental condition).

This proportion is informative and precise. If you want to classify the effect size in more general terms, you should take the square root of eta^2^ to obtain *eta*. As a measure of association, eta can be interpreted with the following rules of thumb:

-   0.1 = small or weak effect,

-   0.3 = medium-sized or moderate effect,

-   0.5 = large or strong effect.

### Between-groups variance and within-groups variance {#between-variance}

For a better understanding of eta^2^ and the statistical test of an analysis of variance model, we have to compare the individual scores to the group averages and to the overall average. Figure \@ref(fig:anova-between) adds overall average willingness to donate to the plot (horizontal black line) with participants' scores and average experimental group scores (coloured horizontal lines).

```{r anova-between, fig.pos='H', fig.align='center', fig.cap="Which part of score differences tells us about the differences between groups?", echo=FALSE, screenshot.opts = list(delay = 5), dev="png", out.width="540px"}
# Goal: Illustrate that between-groups variance represents differences between group means and the grand mean. And that it is a (smaller or larger) proportion of total variance.
# App anova-means: Generate 4 random observations from a normally distributed population with mean 6.4, sd = 1 (Clooney), 4 observations from a population N(m = 6.8, sd = 1) (Jolie), and 4 observations from N(m = 3.3, sd = 1) (no endorser). Use colour for the treatment factor (3 levels). Represent observations in a dotplot, each with a separate value on the x axis, clustered by factor level (experimental condition). Display group means as horizontal line segments (coloured by factor level). Display eta^2 for the data. Allow user to change the group means and update the plot, mean (difference) lines, and eta^2.
# Extension/replacement: Add horizontal line for grand mean, vertical red solid double-sided arrows between each observation and the grand mean (total variance), vertical black solid double-sided arrows for each observation between its group mean and the grand mean (between variance), and vertical black dotted double-sided arrows for each observation between the dot and its group mean (within variance). 
knitr::include_app("http://82.196.4.233:3838/apps/anova-between/", height="490px")
```

<A name="question7.1.4"></A>

```{block2, type='rmdquestion'}
4. In Figure \@ref(fig:anova-between), what do the solid red arrows represent? [<img src="icons/2answer.png" width=115px align="right">](#answer7.1.4)
```

<A name="question7.1.5"></A>

```{block2, type='rmdquestion'}
5. What do the solid black arrows represent? [<img src="icons/2answer.png" width=115px align="right">](#answer7.1.5)
```

<A name="question7.1.6"></A>

```{block2, type='rmdquestion'}
6. What do the dotted black arrows in Figure \@ref(fig:anova-between) represent? [<img src="icons/2answer.png" width=115px align="right">](#answer7.1.6)
```

<A name="question7.1.7"></A>

```{block2, type='rmdquestion'}
7. Which arrows relate to effect size eta^2^? Change group means (and press the _Update graph_ button) and describe what happens. [<img src="icons/2answer.png" width=115px align="right">](#answer7.1.7)
```

Let us assume that we have measured willingness to donate for a sample of 12 participants in our study as depicted in Figure \@ref(fig:anova-between). Once we have our data, we first have a look at the percentage of variance that is explained, eta^2^. What does it mean if we say that a percentage of the variance is explained when we interpret eta^2^?

The variance that we want to explain consists of the differences between the scores of the participants on the dependent variable and the overall or grand mean of all outcome scores. Remember that a variance measures deviations from the mean. The dotted black arrows in Figure \@ref(fig:anova-between) express the distances between outcome scores and the grand average. Squaring, summing, and averaging these distances over all observations gives us the total variance in outcome scores.

The goal of our experiment is to explain why some of our participants have a willingness to donate that is far above the grand mean (horizontal black line in Figure \@ref(fig:anova-between)) while others score a lot lower. We hypothesized that participants are influenced by the endorser they have seen. If an endorser has a positive effect, the average willingness should be higher for participants confronted with this endorser.

If we know the group to which a participant belongs---which celebrity she saw endorsing the fundraising campaign---we can use the average outcome score for the group as the predicted outcome for each group member---her willingness to donate due to the endorser she saw. The predicted group scores are represented by the coloured horizontal lines for group means in Figure \@ref(fig:anova-between).

Now what part of the variance in outcome scores (dotted black arrows in Figure \@ref(fig:anova-between)) is explained by the experimental treatment? If we use the experimental treatment as predictor of willingness to donate, we predict that a participant's willingness equals her group average (horizontal coloured line) instead of the overall average (horizontal black line), which we use if we do not take into account the participant's experimental treatment.

So the difference between the overall average and the group average is what we predict and explain by the experimental treatment. This difference is represented by the solid black arrows in Figure \@ref(fig:anova-between). The variance of the predicted scores is obtained if we average the squared sizes of the solid black arrows for all participants. This variance is called the *between-groups variance*.

Playing with the group means in Figure \@ref(fig:anova-between), you may have noticed that eta^2^ is high if there are large differences between group means. In this situation we have high between-groups variance---large black arrows---so we can predict a lot of the variation in outcome scores between participants.

In contrast, small differences between group averages allow us to predict only a small part of the variation in outcome scores. If all group means are equal, we can predict none of the variation in outcome scores because the between-groups variance is zero. As we will see in Section \@ref(anova-model), zero between-groups variance is central to the null hypothesis in analysis of variance.

```{r skipped1, eval = FALSE, echo = FALSE}
# Skipped to simplify the explanation of ANOVA.

### Within-groups variance {#within-variance}

# {r anova-within, fig.pos='H', fig.align='center', fig.cap="How does within-groups variance relate to between-groups variance?"}
# Goal: Sensitize students to the fact that larger population variance creates larger random variance of sample means.
# 3 populations (of willingness to donate scores; arranged vertically, so means can easily be compared) with equal means and variances (N(5.2, 2)?) ; add button to draw a random sample from all three populations (N = 10 per sample) and display as (3 vertically arranged) dotplots ; display the mean (vertical line) and the variance as a number within each plot ; calculate and display the between-groups variance of the three sample means ; allow user to change the population variance (range: [0, 8], initially 2) to see how it relates to random between-groups variance


1. In Figure \@ref(fig:anova-within), samples are drawn from three populations that have the same means and variances. Do the samples have the same means?

2. How does between-groups variance in the samples relate to the variance in the populations?

3. What happens if you set population variance to zero in Figure \@ref(fig:anova-within)?

4. What, do you expect, is within-groups variance in Figure \@ref(fig:anova-within)?

If we draw samples from the same population or from populations with the same means, the sample means can still be different because we draw samples at random. These sample mean differences are due to chance, they do not reflect true differences between the populations.

Random samples from the same population can only have different sample means if there is variation in the population scores. After all, if all people exposed to George Clooney as endorser would have exactly the same willingness to donate, every random sample drawn from these people would contain people with exactly the same willingness to donate. Average willingness can only be exactly the same for all samples.

The variation in scores within a population, for example, all people who would be exposed to George Clooney as endorser, is called _within-groups variance_. within-groups variance gives rise to chance differences between means of sample drawn from the same or identical population.

The amount of variation in population scores is important. Chance differences between sample means are more likely to be larger if within-groups variance is larger. You are more likely to draw some observations far away from the population mean if score variation is larger in the population. Observations far from the mean influence the sample mean strongly, so the means of samples drawn from this population fluctuate more.
```

The experimental treatment predicts that a participant's willingness equals the average willingness of the participant's group. It cannot predict or explain that a participant's willingness score is slightly different from her group mean (the red double-sided arrows in Figure \@ref(fig:anova-between)). *Within-groups variance* in outcome scores is what we cannot predict with our experimental treatment; it is prediction error. In some SPSS output, it is therefore labeled as "Error".

### *F* test on the model {#anova-model}

Average group scores tell us whether the experimental treatment has effects within the sample (Section \@ref(anova-meandiffs)). If the group who saw Angelina Jolie as endorser has higher average willingness to donate than the group who did not see an endorser, we conclude that Angelina Jolie makes a difference in the sample. But how about the population?

```{r skipped2, eval=FALSE, echo=FALSE}
# Skipped to simplify the explanation of ANOVA.

#### Test statistic

{r anova-F}
# Goal: Illustrate that between-groups variance is zero for equal group means and that it increases with larger (population) differences between group means.
# Generate a sample (N = 12) and plot it as in the app anova-between. Include lines (line segments) for grand mean and group means but do not include double-sided arrows. Display between-groups variance instead of eta^2 as a value. Allow users to change the three group means. Adjust current plots and betweengroups variance but don't draw a new sample. 

1. In which situation is between-groups variance zero? Adjust the group means in Figure \@ref(fig:anova-F) to check your answer.
```

If we want to test whether the difference that we find in the sample also applies to the population, we use the null hypothesis that all average outcome scores are equal in the population from which the samples were drawn. In our example, the null hypothesis states that people in the population who would see George Clooney as endorser are on average just as willing to donate as people who would see Angelina Jolie or who would not see a celebrity endorser at all.

```{r skipped3, eval=FALSE, echo=FALSE}
# Skipped to simplify the explanation of ANOVA.

A statistical test requires a single number that expresses how close the sample result is to the hypothesis. How can we express the equality of three or more population means in one number?

In an independent-samples _t_ test, it is easy to express the equality of the two group means as one number: Just take the difference of the two means. Subtraction, however, does not work for three or more groups.

It is easy to see that subtraction of three means may yield zero even if the means are not the same. Assume that the Clooney group scores 6 as average willingness to donate, the Jolie group scores 4, and the group without celebrity endorser score 2. If we subtract in this order, the result is 6 - 4 - 2 = 0. But the differences between means is not zero!

Instead of subtraction, 
```

We use the variance in group means as the number that expresses the differences between group means. If all groups have the same average outcome score, the between-groups variance is zero. The larger the differences, the larger the between-groups variance (see Section \@ref(between-variance)).

```{r skipped4, eval=FALSE, echo=FALSE}
# Skipped to simplify the explanation of ANOVA.

#### Chance differences in samples

#{r anova-Fratio}
# Goal: Sensitize student to importance of ratio between groups over within-groups variance for rejecting the null hypothesis of equal population means.
# Generate a sample (N = 12) and plot it as in the app anova-F. Display between-groups variance and within-groups variances a pie chart. Display the F and p values. Allow users to change both variances and adjust the dotplot and pie chart accordingly. 

1. How should you adjust between-groups variance and within-groups variance to get more convincing differences between the average group scores in Figure \@ref(fig:anova-Fratio)?

2. What happens to the F test statistic and its p value if differences are more convincing?
```

We cannot just use the between-groups variance as the test statistic because we have to take into account chance differences between sample means. Even if we draw different samples from the same population, the sample means will be different because we draw samples at random. These sample mean differences are due to chance, they do not reflect true differences between groups in the population.

We have to correct for chance differences and this is done by taking the ratio of between-groups variance over within-groups variance. This ratio gives us the relative size of observed differences between group means over group mean differences that we expect by chance.

Our test statistic, then, is the ratio of two variances: between-groups variance and within-groups variance. The *F* distribution approximates the sampling distribution of the ratio of two variances, so we can use this probability distribution to test the significance of the group mean differences we observe in our sample.

Long story short: We test the null hypothesis that all groups have the same population means in an analysis of variance. But behind the scenes, we actually test between-groups variance against within-groups variance. That is why it is called analysis of variance.

### Assumptions for the *F* test in analysis of variance {#anova-assumpt}

There are two important assumptions that we must make if we use the *F* distribution in analysis of variance: (1) independent samples and (2) homogeneous population variances.

#### Independent samples

The first assumption is that the groups can be regarded as independent samples. As in an independent-samples *t* test, it must be possible *in principle* to draw a separate sample for each group in the analysis. Because this is a matter of principle instead of how we actually draw the sample, we have to argue that the assumption is reasonable. We cannot check the assumption against the data.

Here is an example of an argument that we can make. In an experiment, we usually draw one sample of participants and, as a next step, we assign participants randomly to one of the experimental conditions. We could have easily drawn a separate sample for each experimental group. For example, we first draw a participant for the first condition: seeing George Clooney endorsing the fundraising campaign. Next, we draw a participant for the second condition, e.g., Angelina Jolie. The two draws are independent: whomever we have drawn for the Clooney condition is irrelevant to whom we draw for the Jolie condition. Therefore, draws are independent and the samples can be regarded as independent.

Situations where samples cannot be regarded as independent are the same as in the case of dependent/paired-samples *t* tests (see Section \@ref(dependentsamples)). For example, samples of first and second observations in a repeated measurement design should not be regarded as independent samples. Some analysis of variance models can handle repeated measurements but we do not discuss them here.

#### Homogeneous population variances

The *F* test on the null hypothesis of no effect (the nil) in analysis of variance assumes that the groups are drawn from the same population. This implies that they have the same average score on the dependent variable in the population as well as the same variance of outcome scores. The null hypothesis tests the equality of population means but we must assume that the groups have equal dependent variable variances in the population.

We can use a statistical test to decide whether or not the population variances are equal (homogeneous). This is Levene's *F* test, which is also used in combination with independent samples *t* tests. The test's null hypothesis is that the population variances of the groups are equal. If we do *not* reject the null hypothesis, we decide that the assumption of equal population variances is plausible.

The assumption of equal population variances is less important if group samples are more or less of equal size (a balanced design, see Section \@ref(balanced)). We use a rule of thumb that groups are of equal size if the size of the largest group is less than 10% (of the largest group) larger than the size of the smallest group. If this is the case, we do not care about the assumption of homogeneous population variances.

### Which groups have different average scores?

Analysis of variance tests the null hypothesis of equal population means but it does not yield confidence intervals for group means. It does not always tell us which groups score significantly higher or lower.

```{r anova-posthoc, fig.pos='H', fig.align='center', fig.cap="Which groups have different average outcome scores in the population? The _p_ values belong to independent-samples _t_ tests on the means of two groups.", echo=FALSE, screenshot.opts = list(delay = 5), dev="png", out.width="525px"}
# Goal: Sensitize students to the need for and problems with post-hoc tests.
# Generate and display a sample as in the app anova-means with option to change
# group means.
# Display the model F test and the results of three t-tests (without Bonferroni
# correction) for pairwise comparisons (if feasible: as labels to the vertical
# arrows between group means in the plot).
knitr::include_app("http://82.196.4.233:3838/apps/anova-posthoc/", height="485px")
```

<A name="question7.1.8"></A>

```{block2, type='rmdquestion'}
8. Does the _F_ test in analysis of variance tell us which groups have significantly different average population outcome scores? Can we have the same _F_ test result with different sets of group means? Adjust group means in Figure \@ref(fig:anova-posthoc) to demonstrate your answer. [<img src="icons/2answer.png" width=115px align="right">](#answer7.1.8)
```

<A name="question7.1.9"></A>

```{block2, type='rmdquestion'}
9. Is it possible that the _F_ test is statistically significant but none of the _t_ tests that compare groups one by one? Can you obtain this situation in Figure \@ref(fig:anova-posthoc)? [<img src="icons/2answer.png" width=115px align="right">](#answer7.1.9)
```

<A name="question7.1.10"></A>

```{block2, type='rmdquestion'}
10. Is it okay that we apply both an _F_ test and several _t_ tests to the same group differences? [<img src="icons/2answer.png" width=115px align="right">](#answer7.1.10)
```

If the *F* test is statistically significant, we reject the null hypothesis that all groups have the same population mean on the dependent variable. In our current example, we reject the null hypothesis that average willingness to donate is equal for people who saw George Clooney, Angelina Jolie, or no endorser for the fund raiser. In other words, we *reject* the null hypothesis that the endorser does *not* matter to willingness to donate.

#### Pairwise comparisons as post-hoc tests

With a statistically significant *F* test for the analysis of variance model, several questions remain to be answered. Does an endorser increase or decrease the willingness to donate? Are both endorsers equally effective? The *F* test does not provide answers to these questions. We have to compare groups one by one to see which condition (endorser) is associated with a higher level of willingness to donate.

In a pairwise comparison, we have two groups, for instance, participants confronted with George Clooney and participants who did not see a celebrity endorse the fund raiser. We want to compare the two groups on a numeric dependent variable, namely their willingness to donate. An independent-samples *t* test is appropriate here.

With three groups, we can make three pairs: Clooney versus Jolie, Clooney versus nobody, and Jolie versus nobody. We have to execute three *t* tests on the same data. We already know that there are most likely differences in average scores, so the *t* tests are executed after the fact, in Latin *post hoc*. Hence the name *post-hoc tests*.

Applying more than one test to the same data increases the probability of finding at least one statistically significant difference even if there are no differences at all in the population. Section \@ref(cap-chance) discussed this phenomenon as capitalization on chance and it offered a way to correct for this problem, namely Bonferroni correction. We ought to apply this correction to the independent-samples *t* tests that we execute if the analysis of variance *F* test is statistically significant.

The Bonferroni correction divides the significance level by the number of tests that we do. In our example, we do three *t* tests on pairs of groups, so we divide the significance level of five per cent by three. The resulting significance level for each *t* test is .0167. If a *t* test's *p* value is below .0167, we reject the null hypothesis, but we do not reject it otherwise.

#### Two steps in analysis of variance

Analysis of variance, then, consists of two steps. In the first step, we test the general null hypothesis that all groups have equal average scores on the dependent variable in the population. If we cannot reject this null hypothesis, we have too little evidence to conclude that there are differences between the groups. Our analysis of variance stops here, although it is recommended to report the confidence intervals of the group means to inform the reader. Perhaps our sample was just too small to reject the null hypothesis.

If the *F* test is statistically significant, we proceed to the second step. Here, we apply independent-samples *t* tests with Bonferroni correction to each pair of groups to see which groups have significantly different means. In our example, we would compare the Clooney and Jolie groups to the group without celebrity endorser to see if celebrity endorsement increases willingness to donate to the fund raiser, and, if so, how much. In addition, we would compare the Clooney and Jolie groups to see if one celebrity is more effective than the other.

#### Contradictory results

It may happen that the *F* test on the model is statistically significant but none of the post-hoc tests is statistically significant. This mainly happens when the *p* value of the *F* test is near .05. Perhaps the correction for capitalization on chance is too strong; this is known to be the case with the Bonferroni correction. Alternatively, the sample can be too small for the post-hoc test. Note that we have fewer observations in a post-hoc test than in the *F* test because we only look at two of the groups.

This situation illustrates the limitations of null hypothesis significance tests (Chapter \@ref(crit-discus)). Remember that the 5 per cent significance level remains an arbitrary boundary and statistical significance depends a lot on sample size. So do not panic if the *F* and *t* tests have contradictory results.

A statistically significant *F* test tells us that we may be quite confident that at least two group means are different in the population. If none of the post-hoc *t* tests is statistically significant, we should note that it is difficult to pinpoint the differences. Nevertheless, we should report the sample means of the groups (and their standard deviations) as well as the confidence intervals of their differences as reported in the post-hoc test. The two groups that have most different sample means are most likely to have different population means.

### Answers {.unnumbered}

<A name="answer7.1.1"></A>

```{block2, type='rmdanswer'}
Answer to Question 1. 

* The double-sided vertical arrows represent the differences in average
willingness to donate across the three experimental conditions: exposure to
Clooney as endorser, Jolie as endorser, or no celebrity endorser. [<img src="icons/2question.png" width=161px align="right">](#question7.1.1)
```

<A name="answer7.1.2"></A>

```{block2, type='rmdanswer'}
Answer to Question 2. 

* Yes, a celebrity endorser seems to matter to participants’ willingness to donate.
* Overall, participants who saw celebrities Jolie or Clooney have higher willingness to donate (the blue and orange dots) than those who did not see a celebrity endorser (the green dots). Average willingness to donate is clearly higher for participants who saw celebrities Jolie (blue line) or Clooney (orange line) than for the control group (green line), who did not see a celebrity endorser. [<img src="icons/2question.png" width=161px align="right">](#question7.1.2)
```

<A name="answer7.1.3"></A>

```{block2, type='rmdanswer'}
Answer to Question 3. 

* The more different the group means, the larger the red arrows, the larger the
between-groups variance, the larger eta^2^.
* For those of you who love the details: Between-groups variance squares the
distances (actually the distances between group means and the grand mean but
that is not relevant here) before taking the average. As a consequence, a long
red arrow contributes more to between-groups variance than two short arrows
that are just as long as the long arrow if they are summed. This is the reason
that eta^2^ increases if the group in the middle moves closer to the top
group (or, from some point, closer to the bottom group). [<img src="icons/2question.png" width=161px align="right">](#question7.1.3)
```

<A name="answer7.1.4"></A>

```{block2, type='rmdanswer'}
Answer to Question 4. 

![](figures/S7_1Q4.png)

* The solid red arrows represent the difference between an individual score
and the average score of the group to which the individual belongs. For
example, the left-most orange dot represents the willingness score of a
participant who was exposed to Clooney as endorser. The orange line represents
the average willingness score of the participants who were exposed to Clooney.
The red arrow is the difference between the individual's willingness score and
the mean score of its group. Squaring, summing, and averaging the solid red
arrows (within-groups differences) yields the within-groups variance. [<img src="icons/2question.png" width=161px align="right">](#question7.1.4)
```

<A name="answer7.1.5"></A>

```{block2, type='rmdanswer'}
Answer to Question 5. 

* The solid black arrows represent the difference between an individual's group
score, for instance, the average willingness score of all participants who were
exposed to Clooney, and the average score of all participants (the grand or
overall mean).
* If we square, sum, and average these differences, we get the variance of
group means, which is called the between-groups variance. [<img src="icons/2question.png" width=161px align="right">](#question7.1.5)
```

<A name="answer7.1.6"></A>

```{block2, type='rmdanswer'}
Answer to Question 6. 

* The dotted black arrows represent the difference between individual
willingness scores and the average willingness scores of all participants.
Square, sum, and average them to get the overall variance. Take the square
root of the variance to obtain the overall standard deviation. [<img src="icons/2question.png" width=161px align="right">](#question7.1.6)
```

<A name="answer7.1.7"></A>

```{block2, type='rmdanswer'}
Answer to Question 7. 

* The solid black arrows relate to eta^2^. Eta^2^ is larger if the
differences between the group means are larger. The solid black arrows express these differences.

![](figures/S7_1Q7.png)

* If you decrease the differences between the group means, eta^2^
decreases and the solid black arrows become smaller. The red arrows (differences
between scores and their group means) remain the same.
* The dotted black arrows change but some become longer and others become
shorter if you decrease the differences between the group means, so the
dotted black arrows are not clearly related to eta^2^. [<img src="icons/2question.png" width=161px align="right">](#question7.1.7)
```

<A name="answer7.1.8"></A>

```{block2, type='rmdanswer'}
Answer to Question 8. 

* If the _F_ test in analysis of variance is statistically significant, we
reject the null hypothesis that all groups have the same mean score in the
population. If we have more than two groups, as in the example, we do not know
which groups score higher and which groups score lower in the population. We
need post-hoc tests to find that out.
* Because the _F_ test does not tell us which group has a higher or lower score,
we can have the same _F_ value for different situations. For example, exchange
the means of two groups. You will get exactly the same _F_ value (in this
balanced design). [<img src="icons/2question.png" width=161px align="right">](#question7.1.8)
```

<A name="answer7.1.9"></A>

```{block2, type='rmdanswer'}
Answer to Question 9. 

* Yes, this is possible. For example, set the mean score of the No Endorser
group at 4.5, while you leave the Clooney and Jolie averages at 6.4 and 6.8.
* A _t_ test uses only two of the three groups, so the total number of
observations is lower and, therefore, test power is lower and the null
hypothesis is more difficult to reject.
* Note that this situation mainly occurs if the _F_ test is just significant
(slightly below .05). This illustrates that the .05 threshold is an artificial
boundary. [<img src="icons/2question.png" width=161px align="right">](#question7.1.9)
```

<A name="answer7.1.10"></A>

```{block2, type='rmdanswer'}
Answer to Question 10. 

* This is okay provided that we correct for capitalization on chance (see Section \@ref(cap-chance)). [<img src="icons/2question.png" width=161px align="right">](#question7.1.10)
```

## One-Way Analysis of Variance in SPSS {#onewaySPSS}

### Instructions

```{r SPSS1way, echo=FALSE, out.width="640px", fig.pos='H', fig.align='center', fig.cap="(ref:1waySPSS)", dev="png", screenshot.opts = list(delay = 5)}
knitr::include_url("https://www.youtube.com/embed/_2MmzWRcm2k", height = "360px")
# Execute one-way analysis of variance in SPSS with Analyze > Compare Means > One-Way ANOVA {or use general Linear Model also for one-way? (MCRS probably uses Compare Means because it only disucsses one-way ANOVA) - between & within variance not labeled as such!).
# Goal: association as level differences between three or more groups: Does the endorser matter to the level of willingness to donate to a fund raiser?
# Example: donors.sav, outcome is willingness to donate (post), predictor (grouping variable) is endorser (0, 1).
# Technique: one-way ANOVA.
# SPSS menu: Compare Means ; post hoc: Bonferroni ; options: descriptives, homogeneity of variance test, means plot.
# Paste & Run.
# Check assumptions: F test homogeneous population variances or groups of equal size ; post-hoc t tests: each group more than 30 observations or normally distributed.
# Interpret output: F test on the null hypothesis that all groups have equal population means - point out between groups sum of squares? ; post-hoc t test for pairwise comparison of (population) means ; test result and significance, confidence interval.
```

### Exercises

<A name="question7.2.1"></A>

```{block2, type='rmdquestion'}
1. How does celebrity endorsement affect the willingness to donate and is one celebrity more effective than the other? Use the data in [donors.sav](http://82.196.4.233:3838/data/donors.sav). [<img src="icons/2answer.png" width=115px align="right">](#answer7.2.1)
```

<A name="question7.2.2"></A>

```{block2, type='rmdquestion'}
2. The data set [smokers.sav](http://82.196.4.233:3838/data/smokers.sav) contains information on smoking behaviour and attitude towards smoking for a random sample of adults. Does the attitude towards smoking differ among smokers, former smokers, and non-smokers (variable: *status3*)? [<img src="icons/2answer.png" width=115px align="right">](#answer7.2.2)
```

### Answers {.unnumbered}

<A name="answer7.2.1"></A>

```{block2, type='rmdanswer'}
Answer to Exercise 1. 

SPSS syntax:  
  
\* Check data.  
FREQUENCIES VARIABLES=willing_post endorser  
  /ORDER=ANALYSIS.  
\* One-way analysis of variance.  
ONEWAY willing_post BY endorser  
  /ES=OVERALL
  /STATISTICS DESCRIPTIVES HOMOGENEITY   
  /PLOT MEANS  
  /MISSING ANALYSIS  
  /POSTHOC=BONFERRONI ALPHA(0.05).  
\* Note: ES=OVERALL gives eta^2^ (SPSS version 27 and higher)
  
Check data:  
  
There are no impossible values on the variables.  
  
Check assumptions:  
  
The three groups are more or less of equal size: The largest difference is 4
participants, which is less than ten per cent of the largest group (*N* = 49).
Anyway, we may assume equal population variances, Levene *F* (2, 140) = .02, *p* =
.978. Note that the results of this test can be slightly different in
different versions of SPSS.
  
Interpret the results:  
  
* Willingness to donate depends on the endorsing celebrity. There is a statistically significant difference between average willingness to donate for the three endorsers, *F* (2, 140) = 7.44, *p* = .001, eta^2^ = 0.10, 95% CI [0.02, 0.19].  
* People are more willing to donate if they have seen Clooney (*M* = 4.99, *SD* = 1.64) or Jolie (*M* = 4.95, *SD* = 1.63) endorse the fund raiser than people who do not see a celebrity endorser (*M* = 3.87, *SD* = 1.47). The differences between, on the one hand, no celebrity endorser and, on the other hand, Clooney (*p* = .002, 95% CI [-1.91, -0.33]) or Jolie (*p* = .004, 95% CI [-1.88, -0.29]) are statistically significant and range between circa 0.3 and 1.9 with 95% confidence in the population.
* However, there is not a substantial or statistically significant difference between Clooney and Jolie with respect to their effect on willingness to donate (mean difference = 0.04, *p* = 1.000, 95% CI [-0.74, 0.81]); participants seeing Clooney may just as well have higher as lower willingness to donate than participants seeing Jolie.
  
Instead of reporting the _F_ test result in the text, the ANOVA table can be included. 

Note that eta^2^ must be calculated by hand in SPSS version 26 or older: Divide the sum of squares of the main effect (between groups) by the total sum of squares. The confidence interval for eta^2^ cannot be calculated by hand. [<img src="icons/2question.png" width=161px align="right">](#question7.2.1)
```

<A name="answer7.2.2"></A>

```{block2, type='rmdanswer'}
Answer to Exercise 2. 

SPSS syntax:  
  
\* Check data.  
FREQUENCIES VARIABLES=attitude status3  
  /ORDER=ANALYSIS.  
\* One-way analysis of variance.  
ONEWAY attitude BY status3  
  /ES=OVERALL
  /STATISTICS DESCRIPTIVES HOMOGENEITY   
  /PLOT MEANS  
  /MISSING ANALYSIS  
  /POSTHOC=BONFERRONI ALPHA(0.05).  
\* Note: ES=OVERALL gives eta^2^ (SPSS version 27 and higher)
  
Check data:  
  
There are no impossible scores on the two variables.  
  
Check assumptions:  
  
The Levene test is not statistically significant, *F* (2, 82) = 2.82, *p* = .066,
so we assume that smoking attitude for the three groups have equal population
variances. Note that different versions of SPSS may give slightly different
results, but the test is never statistically significant.
  
Interpret the results:  
  
In the sample, former smokers have a much more negative attitude towards smoking (*M* = -1.69, *SD* = 1.71) than non-smokers (*M* = 0.64, *SD* = 1.17) and smokers (*M* = 0.80, *SD* = 1.67). 

With 95% confidence, the attitude towards smoking for former smokers is on average at least 1.35 points more negative on a scale from -5 to +5 than for non-smokers and the difference can be as large as 3.3 points in the population (mean difference = -2.33, *p* < .001, 95% CI [-3.31, -1.35]). Similarly, former smokers have an attitude that is on average 1.3 to 3.7 points more negative than smokers (mean difference = -2.49, *p* < .001, 95% CI [-3.66, -1.32]). The difference in attitude towards smoking between smokers and non-smokers in the population is not clear. Smokers can on average be up to 0.78 points more negative about smoking than non-smokers, but they can also be up to 1.09 points more positive (mean difference = 0.16, *p* = 1.000, 95% CI [-0.78, 1.09]). We need a larger sample to be more specific about the difference between smokers and non-smokers.

Due to the differences between on the one hand former smokers and on the other hand smokers and non-smokers, average attitude towards smoking scores are significantly different across smoking status, *F* (2, 82) = 18.85, *p* < .001, eta^2^ = 0.31 (SPSS yields 0.31498, which is rounded to 0.315). We have sufficient evidence to conclude that former smokers have a more negative attitude towards smoking than the other groups in the population of adults. The differences are substantial (a strong effect, _eta_ = square root of 0.31 = 0.56).

Note that eta^2^ must be calculated by hand in SPSS version 26 or older: Divide the sum of squares of the main effect (between groups) by the total sum of squares. The confidence interval for eta^2^ cannot be calculated by hand. [<img src="icons/2question.png" width=161px align="right">](#question7.2.2)
```

## Different Means for Two Factors

The participants in the experiment do not only differ because they see different endorsers in the charity video. In addition, there are differences according to sex: female versus male participants. Does participant's sex matter to the effect of the endorser on willingness to donate?

```{r anova-twoway, fig.pos='H', fig.align='center', fig.cap="How do group means inform us about (main) effects in analysis of variance?", echo=FALSE, screenshot.opts = list(delay = 5), dev="png", out.width="775px"}
# Goal: Illustrate that different main effects merely use means of different
# groupings.
# Similar to app anova-between but with a double classification of cases
# (according to endorser and sex) and te option to display means of different
# groupings.
# Generate 6 sets of 2 (random) observations (from a normally distributed
# population with mean runif(3, 7) and sd = 1). Assign the groups to the
# experimental treatment factor (endorser, 3 levels) and sex factor (2 levels).
# Represent observations in a dotplot with treatment as dot colour and sex as
# dot shape, each observation with a separate value on the x axis, ordered by
# factor levels.
# Display the grand mean as a horizontal line. 
# Allow the user to select (display) group means on one of the two factors or
# both. On selection, add group means as horizontal lines (coloured for the 3
# levels factor and different line styles for 2 levels factor) with between
# group variation indicated by double-sided arrows between group mean and grand
# mean for each dot (between). Add subgroup (endorser by sex) means if both
# factors are selected.
knitr::include_app("http://82.196.4.233:3838/apps/anova-twoway/", height="408px")
```

<A name="question7.3.1"></A>

```{block2, type='rmdquestion'}
1. How does an analysis of variance test the effect of endorser on willingness to donate with the data displayed in Figure \@ref(fig:anova-twoway)? Select the endorser factor to check your answer. [<img src="icons/2answer.png" width=115px align="right">](#answer7.3.1)
```

<A name="question7.3.2"></A>

```{block2, type='rmdquestion'}
2. Compare a plot with the endorser factor selected to a plot with the sex factor selected. Which effect on willingness to donate is probably stronger: the effect of endorser or of sex? Motivate your answer, for example, using the grey arrows in the plots. [<img src="icons/2answer.png" width=115px align="right">](#answer7.3.2)
```

<A name="question7.3.3"></A>

```{block2, type='rmdquestion'}
3. Where do you expect the group means to show up in the graph if you select both the Endorser and Sex check boxes? [<img src="icons/2answer.png" width=115px align="right">](#answer7.3.3)
```

In the preceding section, we have looked at the effect of a single factor on willingness to donate, namely, the endorser to whom participants are exposed. Thus, we take into account two variables: one independent variable and one dependent variable. This is an example of *bivariate analysis*.

Usually, however, we expect an outcome to depend on more than one variable. Willingness to donate does not depend only on the celebrity endorsing a fundraising campaign. It is easy to think of more factors, such as a person's available budget, her personal level of altruism, and so on.

It is straightforward to include more factors in an analysis of variance. These can be additional experimental treatments in the context of an experiment as well as participant characteristics that are not manipulated by the researcher. For example, we may hypothesize that females are generally more charitable than males.

### Two-way analysis of variance {#anova2way}

If we use one factor, the analysis is called one-way analysis of variance. With two factors, it is called two-way analysis of variance, and with three factors... well, you probably already guessed that name.

A two-way analysis of variance using a factor with three levels, for instance, exposure to three different endorsers, and a second factor with two levels, for example, female versus male, is called a 3x2 (say: three by two) factorial design.

### Balanced design {#balanced}

In analysis of variance with two or more factors, it is quite nice if the factors are statistically independent from one another. In other words, it is nice if the scores on one factor are not associated with scores on another factor. This is called a *balanced design*.

In an experiment, we can ensure that factors are independent if we have the same number of participants in each combination of levels on all factors. In other words, a factorial design is balanced if we have the same number of observations in each subgroup. A subgroup contains the participants that have the same level on both factors just like a cell in a contingency table.

```{r anova-balanced, echo=FALSE}
# Table for a balanced 3x2 factorial design.
# Create data.
df <- data.frame(Female = rep(2, 3), Male = rep(2, 3))
row.names(df) <- c("Clooney", "Jolie", "No endorser")
# Display table.
knitr::kable(df, booktabs = TRUE, caption = "Number of observations per subgroup in a balanced 3x2 factorial design.") %>%
  kable_styling(font_size = 12, full_width = F, position = "float_right",
                latex_options = c("HOLD_position"))
# Cleanup.
rm(df)
```

Table \@ref(tab:anova-balanced) shows an example of a balanced 3x2 factorial design. Each subgroup (cell) contains two participants (cases). Equal distributions of frequencies across columns or across rows indicate statistical independence. In the example, the distributions are the same across columns (and rows), so the factors are statistically independent.

In practice, it may not always be possible to have exactly the same number of observations for each subgroup. A participant may drop out from the experiment, a measurement may go wrong, and so on. If the numbers of observations are more or less the same for all subgroups, the factors are nearly independent, which is okay. We can use the same rule of thumb for a balanced design as for the conditions of an *F* test in analysis of variance: If the size of the smallest subgroup is less than ten per cent smaller than the size of the largest group, we call a factorial design balanced.

A balanced design is nice but not necessary. Unbalanced designs can be analyzed but estimation is more complicated (a problem for the computer, not for us) and the assumption of equal population variances for all groups (Levene's *F* test) is more important (a problem for us, not for the computer) because we do not have equal group sizes. Note that the requirement of equal group sizes applies to the *subgroups* in a two-way analysis of variance. With a balanced design, we ensure that we have the same number of observations in all subgroups, so we are on the safe side.

### Main effects in two-way analysis of variance {#maineffects}

A two-way analysis of variance tests the effects of both factors on the dependent variable in one go. It tests the null hypothesis that participants exposed to Clooney have the same average willingness to donate in the population as participants exposed to Jolie or those who are not exposed to an endorser. At the same time, it tests the null hypothesis that females and males have the same average willingness to donate in the population.

```{r anova-meansplot2, echo=FALSE, fig.pos='H', fig.align='center', fig.cap="Means plots for the main effects of endorser and sex on willingness to donate. As a reading instruction, effects of endorsers and of being female are represented by arrows.", out.width="50%", fig.asp=0.8, fig.show='hold'}
# Insert means plot for celebrity endorsement example.
d <- data.frame(
  endorser = factor(c("Nobody","Nobody","Clooney","Clooney","Jolie","Jolie"), 
                    levels = c("Nobody","Clooney","Jolie")),
  sex = factor(rep(c("Women", "Men"), 3)),
  willingness_av = c(4.5, 3, 6.5, 5, 8.5, 7), const = 1)
library(ggplot2)
d2 <- d %>% group_by(endorser) %>%
  summarise(willing = mean(willingness_av),
            const = max(const))
# Plot for main effect endorser.
ggplot(d2, aes(endorser, willing)) + 
    geom_point(size = 3, color=brewercolors["Blue"]) + 
    geom_line(aes(group = const), size = 1, color=brewercolors["Blue"]) + 
    geom_segment(aes(x = 1, xend = 3, y = willing[[1]], yend = willing[[1]]),
               linetype = "dashed", color = "black") +
    geom_segment(aes(x = 2, xend = 2, y = willing[[1]], yend = (willing[[2]] - 0.1)),
               color = "darkgrey",
               arrow = arrow(length = unit(2,"mm"),
                                 # ends = "both",
                                 type = "closed")) +
    geom_text(aes(x = 2.02, y = (willing[[1]] + willing[[2]])/2,
            label = "Clooney effect",
            hjust = 0), color = "darkgrey", size =5
            ) +
    geom_segment(aes(x = 3, xend = 3, y = willing[[1]], yend = (willing[[3]] - 0.1)),
               color = "darkgrey",
               arrow = arrow(length = unit(2,"mm"),
                                 # ends = "both",
                                 type = "closed")) +
    geom_text(aes(x = 2.98, y = (willing[[1]] + willing[[3]])/2,
            label = "Jolie effect",
            hjust = 1), color = "darkgrey", size =5
            ) +
    theme_general() + 
    theme(text = element_text(size = 18)) +
    scale_y_continuous(limits = c(1, 10), breaks = c(1, 5, 10)) + 
    labs(x = "Endorser", y = "Average willingness to donate")
# Plot for main effect sex.
d %>% group_by(sex) %>%
  summarise(willing = mean(willingness_av),
            const = max(const)) %>%
  ggplot(aes(sex, willing)) + 
    geom_point(size = 3, color=brewercolors["Blue"]) + 
    geom_line(aes(group = const), size = 1, color=brewercolors["Blue"]) + 
    geom_segment(aes(x = 1, xend = 2, y = willing[[1]], yend = willing[[1]]),
               linetype = "dashed", color = "black") +
    geom_segment(aes(x = 2, xend = 2, y = willing[[1]], yend = (willing[[2]] - 0.1)),
               color = "darkgrey",
               arrow = arrow(length = unit(2,"mm"),
                                 # ends = "both",
                                 type = "closed")) +
    geom_text(aes(x = 1.98, y = (willing[[1]] + willing[[2]])/2,
            label = "Effect of being female",
            hjust = 1), color = "darkgrey", size =5
            ) +
    theme_general() + 
    theme(text = element_text(size = 18)) +
    scale_y_continuous(limits = c(1, 10), breaks = c(1, 5, 10)) + 
    labs(x = "Sex", y = "Average willingness to donate")
rm(d)
```

The tested effects are main effects because they represent the effect of one factor. They express an overall or average difference between the mean scores of the groups on the dependent variable. The main effect of the endorser factor shows the mean differences for endorser groups if we do not distinguish between females and males. Likewise, the main effect for sex shows the average difference in willingness to donate between females and males without taking into account the endorser to whom they were exposed.

We could have used two separate one-way analyses of variance to test the same effects. Moreover, we could have tested the difference between females and males with an independent-samples *t* test. The results would have been the same (if the design is balanced.) But there is an important advantage to using a two-way analysis of variance, to which we turn in the next section.

### Answers {.unnumbered}

<A name="answer7.3.1"></A>

```{block2, type='rmdanswer'}
Answer to Question 1. 

* Analysis of variance tests the variance (variation, spread) in the mean
scores on the dependent variable among groups. If the group means are more widely
apart, the variance of group means is larger, so the _F_ test value is larger and
further away from what we expect if there are no differences among group means
in the population. The variation among group means is larger if they are more
distant from the overall (or grand) mean.
* To test the endorser effect, analysis of variance looks at the mean scores of
groups defined by the celebrity endorser they have seen. It will check the
distance between the three celebrity group means and the grand mean. [<img src="icons/2question.png" width=161px align="right">](#question7.3.1)
```

<A name="answer7.3.2"></A>

```{block2, type='rmdanswer'}
Answer to Question 2. 

* The effect of endorser is probably stronger than the effect of sex because
the observations within an endorser group are less equally distributed around
the grand mean (they are more clustered above or below the grand mean) than
observations within a sex group. Both females and males are found nearly
equally above and below the grand mean, so their group means are close to the
grand mean, and there is little variation in sex group means.

![](figures/S7_3Q2.png)

* The grey arrows represent the distance between group mean and grand mean for each observation. They are much shorter for the difference between females and males than for differences between endorser groups. [<img src="icons/2question.png" width=161px align="right">](#question7.3.2)
```

<A name="answer7.3.3"></A>

```{block2, type='rmdanswer'}
Answer to Question 3. 

* If both boxes are ticked, the scores are grouped by the combination of
endorser and sex. The graph will show the means of subgroups: females exposed to
Clooney, males exposed to Clooney, males exposed to Jolie, and so on. [<img src="icons/2question.png" width=161px align="right">](#question7.3.3)
```

## Moderation: Group-Level Differences that Depend on Context {#moderationanova}

```{r echo=FALSE}
# TBD: the concept of moderation ; picture (sketch as video) as icon
# for moderation
```

In the preceding section, we have analyzed the effects both of endorser and sex on willingness to donate to a fund raiser. The two main effects isolate the influence of endorser on willingness from the effect of sex and the other way around. This assumes that endorser and sex have an effect on their own, a general effect.

We should, however, wonder whether endorser always has the same effect. Even if there is a general effect of endorser on willingness to donate, is this effect the same for females and males? Note that one endorser is a male celebrity who is reputed to be quite attractive to women. The other endorser is a female celebrity with a similar reputation among men. In this situation, shouldn't we expect that one endorser is more effective among female participants and the other among male participants?

If the effect of a factor is different for different groups on another factor, the first factor's effect is *moderated* by the second factor. The phenomenon that effects are moderated is called *moderation*. Both factors are independent variables. To distinguish between them, we will henceforth refer to them as the predictor and the moderator.

With moderation, factors have a combined effect. The context (group score on one factor) affects the effect of the other factor on the dependent variable. The conceptual diagram for moderation expresses the effect of the moderator on the effect of the predictor as an arrow pointing at another arrow. Figure \@ref(fig:anova-diagram) shows the conceptual diagram for participant's sex moderating the effect of endorsing celebrity on willingness to donate.

```{r anova-diagram, echo=FALSE, fig.pos='H', fig.align='center', fig.cap="Conceptual diagram of moderation.", fig.asp=0.3}
library(ggplot2)
# Create coordinates for the variable names.
variables <- data.frame(x = c(0.3, 0.5, 0.7), 
                        y = c(.1, .3, .1),
                        label = c("Endorser", "Sex", "Willingness"))
ggplot(variables, aes(x, y)) + 
  geom_segment(aes(x = x[1], y = y[1], xend = x[3] - 0.05, yend = y[1]), arrow = arrow(length = unit(0.04, "npc"), type = "closed")) + 
  geom_segment(aes(x = x[2], y = y[2], xend = x[2], yend = y[1]), arrow = arrow(length = unit(0.04, "npc"), type = "closed")) +
  geom_label(aes(label=label)) + 
  coord_cartesian(xlim = c(0.2, 0.8), ylim = c(0, 0.4)) +
  theme_void()
#Cleanup.
rm(variables)
```

### Types of moderation

Moderation as different effects for different groups is best interpreted using a cross-tabulation of group means, which is visualized as a means plot. In a group means table, the **Totals** row and column contain the means for each factor separately, for example the means for males and females (factor sex) or the means for the endorsers (factor endorser). These means represent the main effects. In contrast, the means in the cells of the table are the means of the subgroups, which represent moderation. Draw them in a means plot for easy interpretation.

In a means plot, we use the groups of the predictor on the horizontal axis, for example, the three endorsers. The average score on the dependent variable is used as the vertical axis. Finally, we plot the average scores for every predictor-moderator group, for instance, an endorser-sex combination, and we link the means that belong to the same moderator group, for example, the means for females and the means for males (Figure \@ref(fig:anova-moderation)).

```{r anova-moderation, fig.pos='H', fig.align='center', fig.cap="How can we recognize main effects and moderation in a means plot?", echo = FALSE, screenshot.opts = list(delay = 5), dev="png", out.width="775px"}
# Goals: Learn to recognize different types of moderation, visually distinguishing between main effects (differences) and moderation (different differences.
# Create a means plot with 3x2 means, willingness on Y axis, endorser (nobody, Clooney, Jolie) on X axis, and different colours for sex. Initial means have main effects but no interaction effect.Connect the means per sex by line segments. Link the female & male mean for the same group by a vertical double-sided arrow. Allow user to change all six means (if possible, by dragging them vertically?). Display marginal (total) means for each sex and endorser.
# Initial means.
# d <- data.frame(endorser = factor(c("Nobody","Clooney","Jolie","Nobody","Clooney","Jolie"), levels = c("Nobody","Clooney","Jolie")), sex = as.factor(c(rep("male", 3), rep("female", 3))), willingness_av = c(3, 5, 7, 4.5, 6.5, 8.5))
knitr::include_app("http://82.196.4.233:3838/apps/anova-moderation/", height="305px")
```

<A name="question7.4.1"></A>

```{block2, type='rmdquestion'}
1. Does the plot in Figure \@ref(fig:anova-moderation) display a main effect of the factor sex? Motivate your answer with numbers from the table and/or the lines in the plot. If sex has a main effect in this sample, describe the effect: What is the difference between women and men here? [<img src="icons/2answer.png" width=115px align="right">](#answer7.4.1)
```

<A name="question7.4.2"></A>

```{block2, type='rmdquestion'}
2. Is there a main effect of endorser? Again, motivate your answer and describe the effect if there is one. [<img src="icons/2answer.png" width=115px align="right">](#answer7.4.2)
```

<A name="question7.4.3"></A>

```{block2, type='rmdquestion'}
3. Is the effect of participant's sex the same for all three types of endorser in Figure \@ref(fig:anova-moderation)? How can you tell? [<img src="icons/2answer.png" width=115px align="right">](#answer7.4.3)
```

<A name="question7.4.4"></A>

```{block2, type='rmdquestion'}
4. Adjust the means in such a way that the effect of sex is different for different endorsers. In other words, adjust the means such that sex moderates the effect of endorser on willingness to donate. [<img src="icons/2answer.png" width=115px align="right">](#answer7.4.4)
```

Moderation happens a lot in communication science for the simple reason that the effects of messages are stronger for people who are more susceptible to the message. If you know more people who have adopted a new product or a healthy/risky lifestyle, you are more likely to be persuaded by media campaigns to also adopt that product or lifestyle. If you are more impressionable in general, media messages are more effective.

#### Effect strength moderation

Moderation refers to contexts that strengthen or diminish the effect of, for instance, a media campaign. Let us refer to this type of moderation as *effect strength moderation*. In our current example, we would hypothesize that the effect of George Clooney as an endorser is stronger for female participants than for male participants.

In analysis of variance, effects are differences between average outcome scores. The effect of Clooney on willingness to donate, for instance, is the difference between the average willingness score of participants exposed to Clooney and the average score of participants who were not exposed to a celebrity endorser.

Different "Clooney effects" for female and male participants imply different differences! The difference in average willingness scores between females exposed to Clooney and females who are not exposed to an endorser is different from the difference in average scores for males. We have four subgroups with average willingness scores that we have to compare. We have six subgroups if we also include endorsement by Angelina Jolie.

```{r anova-effectstrengthmod, echo=FALSE, out.width="70%", fig.asp=0.6, fig.pos='H', fig.align='center', fig.cap="Moderation as a stronger effect within a particular context."}
# Add means plot with Clooney, Jolie, and Nobody on the x axis, willingness to donate on the y axis, and separate (coloured) lines linking the mean scores for females and males. In the plot, the average for females*Clooney is much higher than males*Clooney but the reverse does not apply to Jolie.
d <- data.frame(endorser = factor(c("Nobody","Clooney","Jolie","Nobody","Clooney","Jolie"), levels = c("Clooney","Nobody","Jolie")), sex = as.factor(c(rep("male", 3), rep("female", 3))), willingness_av = c(3, 5, 5, 4.5, 9, 6.5))
library(ggplot2)
ggplot(d, aes(endorser, willingness_av, colour = sex)) + 
  geom_point(size = 3) + 
  geom_line(aes(group = sex), size = 1) + 
  geom_segment(aes(x = 0.8, xend = 2.2, y = d[5,3], yend = d[5,3]),
               linetype = "dashed", color = "black") +
  geom_segment(aes(x = 2, xend = 2, y = d[4,3], yend = d[5,3]),
               color = "darkgrey",
               arrow = arrow(length = unit(2,"mm"),
                                 # ends = "both",
                                 type = "closed")) +
  geom_text(aes(x = 2.02, y = (d[4,3] + d[5,3])/2,
            label = "Clooney effect\nfor females",
            hjust = 0), color = "darkgrey"
            ) +
  geom_segment(aes(x = 0.8, xend = 2.2, y = d[1,3], yend = d[1,3]),
               linetype = "dashed", color = "black") +
  geom_segment(aes(x = 1, xend = 1, y = d[1,3], yend = (d[2,3] - 0.1)),
               color = "darkgrey",
               arrow = arrow(length = unit(2,"mm"),
                                 # ends = "both",
                                 type = "closed")) +
  geom_text(aes(x = 0.98, y = (d[1,3] + d[2,3])/2,
            label = "Clooney effect\nfor males",
            hjust = 1), color = "darkgrey"
            ) +
  theme_general() + 
  scale_y_continuous(limits = c(1, 10), breaks = c(1, 5, 10)) + 
  labs(x = "Endorser", y = "Average willingness to donate") +
  scale_color_manual(values=c(brewercolors[[1]], brewercolors[[5]]))
rm(d)
```

A means plot is a very convenient tool to interpret different differences. Connect the means of the subgroups by lines that belong to the same group on the factor you use as moderator. Each line in the plot represents the effect differences within one moderator group. If a line goes up or down, predictor groups have different means, so the predictor has an effect within that moderator group. A flat (horizontal) line tells us that there is no effect at all within that moderator group

The distances between the lines show the difference of the differences. If the lines for females and males are parallel, the difference between endorsers is the same for females and males. Then, the effects are the same and there is *no* moderation. In contrast, if the lines are not parallel but diverge or converge, the differences are different for females and males and there is moderation.

A special case of effect strength moderation is the situation in which the effect is absent (zero) in one context and present in another context. A trivial example would be the effect of an anti-smoking campaign on smoking frequency. For smokers (one context), smoking frequency may go down with campaign exposure and the campaign may have an effect. For non-smokers (another context), smoking frequency cannot go down and the campaign cannot have this effect.

Except for trivial cases such as the effect of anti-smoking campaigns on non-smokers, it does not make much sense to distinguish sharply between moderation in which the effect is strengthened and moderation in which the effect is present versus absent. In non-trivial cases, it is very rare that an effect is precisely zero. (See Holbert and Park [-@HolbertConceptualizingOrganizingPositing2019] for a different view on this matter.)

#### Effect direction moderation

In the other type of moderation, the effect in one group is the opposite of the effect in another group. In figure \@ref(fig:anova-effectdirmod1), for example, Clooney increases the average willingness to donate among females in comparison to the group who did not see a celebrity endorser. In contrast, average willingness for male Clooney viewers is lower than the average for males without endorser. Let us call this *effect direction moderation*. Males reverse the Clooney effect that we find for females.

```{r anova-effectdirmod1, echo=FALSE, out.width="70%", fig.asp=0.6, fig.pos='H', fig.align='center', fig.cap="Moderation as a positive  effect in one context and a negative effect in another context."}
# Add means plot with Clooney, Jolie, and Nobody on the x axis, willingness to donate on the y axis, and separate (coloured) lines linking the mean scores for females and males. In the plot, the average for females*Clooney is much higher than males*Clooney but the reverse does not apply to Jolie.
d <- data.frame(endorser = factor(c("Nobody","Clooney","Jolie","Nobody","Clooney","Jolie"), levels = c("Clooney","Nobody","Jolie")), sex = as.factor(c(rep("male", 3), rep("female", 3))), willingness_av = c(5, 3.5, 7, 6, 7.5, 9))
library(ggplot2)
ggplot(d, aes(endorser, willingness_av, colour = sex)) + 
  geom_point(size = 3) + 
  geom_line(aes(group = sex), size = 1) + 
  geom_segment(aes(x = 0.8, xend = 2.2, y = d[5,3], yend = d[5,3]),
               linetype = "dashed", color = "black") +
  geom_segment(aes(x = 2, xend = 2, y = d[4,3], yend = d[5,3]),
               color = "darkgrey",
               arrow = arrow(length = unit(2,"mm"),
                                 # ends = "both",
                                 type = "closed")) +
  geom_text(aes(x = 1.98, y = (d[4,3] + d[5,3])/2,
            label = "Clooney effect\nfor females",
            hjust = 1), color = "darkgrey"
            ) +
  geom_segment(aes(x = 0.8, xend = 2.2, y = d[1,3], yend = d[1,3]),
               linetype = "dashed", color = "black") +
  geom_segment(aes(x = 1, xend = 1, y = d[1,3], yend = (d[2,3] + 0.1)),
               color = "darkgrey",
               arrow = arrow(length = unit(2,"mm"),
                                 # ends = "both",
                                 type = "closed")) +
  geom_text(aes(x = 0.98, y = (d[1,3] + d[2,3])/2,
            label = "Clooney effect\nfor males",
            hjust = 1), color = "darkgrey"
            ) +
  theme_general() + 
  scale_y_continuous(limits = c(1, 10), breaks = c(1, 5, 10)) + 
  labs(x = "Endorser", y = "Average willingness to donate") +
  scale_color_manual(values=c(brewercolors[[1]], brewercolors[[5]]))
rm(d)
```

In an extreme situation, the effect in one group can compensate for the effect in another group if it is about as strong but of the opposite direction (Figure \@ref(fig:anova-effectdirmod2)). Imagine that George Clooney convinces females to donate but discourages males to donate because his charms backfires on men (pure jealousy, perhaps.) Similarly, Angelina Jolie may have opposite effects on females and males.

```{r anova-effectdirmod2, echo=FALSE, out.width="70%", fig.asp=0.6, fig.pos='H', fig.align='center', fig.cap="Moderation as opposite effects in different contexts."}
# Add means plot with Clooney, Jolie, and Nobody on the x axis, willingness to donate on the y axis, and separate (coloured) lines linking the mean scores for females and males. In the plot, the average for females*Clooney is much higher than males*Clooney but the reverse does not apply to Jolie.
d <- data.frame(endorser = factor(c("Nobody","Clooney","Jolie","Nobody","Clooney","Jolie"), levels = c("Clooney","Nobody","Jolie")), sex = as.factor(c(rep("male", 3), rep("female", 3))), willingness_av = c(6, 3, 9, 6, 9, 3))
library(ggplot2)
ggplot(d, aes(endorser, willingness_av, colour = sex)) + 
  geom_point(size = 3) + 
  geom_point(aes(x = endorser, y = 6), size = 3, color = "darkgrey") +
  geom_line(aes(group = sex), size = 1) + 
  geom_segment(aes(x = 0.8, xend = 3.2, y = d[1,3], yend = d[1,3]),
               linetype = "dashed", color = "black") +
  geom_segment(aes(x = 1, xend = 1, y = d[4,3], yend = (d[5,3] - 0.1)),
               color = "darkgrey",
               arrow = arrow(length = unit(2,"mm"),
                                 # ends = "both",
                                 type = "closed")) +
  geom_text(aes(x = 0.98, y = (d[4,3] + d[5,3])/2,
            label = "Clooney effect\nfor females",
            hjust = 1), color = "darkgrey") +
  geom_segment(aes(x = 3, xend = 3, y = d[4,3], yend = d[6,3]),
               color = "darkgrey",
               arrow = arrow(length = unit(2,"mm"),
                                 # ends = "both",
                                 type = "closed")) +
  geom_text(aes(x = 3.02, y = (d[4,3] + d[6,3])/2,
            label = "Jolie effect\nfor females",
            hjust = 0), color = "darkgrey") +
  theme_general() + 
  scale_y_continuous(limits = c(1, 10), breaks = c(1, 5, 10)) + 
  labs(x = "Endorser", y = "Average willingness to donate") + 
  scale_color_manual(values=c(brewercolors[[1]], brewercolors[[5]]))
rm(d)
```

In this situation, the main effect of endorser on willingness to donate is around zero. If we average over females and males, we obtain the means represented by the three grey dots. There is no net difference between Clooney, Jolie, and the condition without an endorser.

This does not mean that the endorser does not matter. On the contrary, the interaction effects tell us that the endorser is effective for one group but counterproductive for another group. The second part of the conclusion is just as important as the first part. The campaigner should avoid decreasing the willingness to donate among particular target groups.

### Testing main and interaction effects

The effect of a single factor is called a main effect, as we learned in Section \@ref(maineffects). A main effect reflects the difference between means for groups within one factor. The main effect of sex, for instance, can be that females are on average more willing to donate than males. A two-way analysis of variance includes two main effects, one for each factor (see Section \@ref(anova2way)), for example a main effect of sex and a main effect of endorser.

For moderation, however, we compare average scores of subgroups, that is, groups that combine a level on one factor with a level on another factor. In Figure \@ref(fig:anova-interaction), we compare average willingness to donate for combinations of endorser and participant's sex. The effect of differences among subgroups on the dependent variable is called an *interaction effect*. Just like a main effect, an interaction effect is tested with an *F* test and its effect size is expressed by eta^2^.

```{r anova-interaction, echo=FALSE, fig.pos='H', fig.align='center', fig.cap="How can we recognize main effects and moderation in a means plot?", screenshot.opts = list(delay = 5), dev="png", out.width="780px"}
# Goals: Recognize effect size and statistical significance of main and interaction effects.
# Exactly the same app as anova-moderation but now add Eta^2 and the _F_ value plus _p_ value of the main and interaction effects.
# Initial means.
# d <- data.frame(endorser = factor(c("Nobody","Clooney","Jolie","Nobody","Clooney","Jolie"), levels = c("Nobody","Clooney","Jolie")), sex = as.factor(c(rep("male", 3), rep("female", 3))), willingness_av = c(3, 5, 7, 4.5, 6.5, 8.5))
knitr::include_app("http://82.196.4.233:3838/apps/anova-interaction/", height="380px")
```

<A name="question7.4.5"></A>

```{block2, type='rmdquestion'}
5. Adjust the means in Figure \@ref(fig:anova-interaction) in such a way that the main effect of endorser and the interaction effect of endorser and sex are statistically significant. [<img src="icons/2answer.png" width=115px align="right">](#answer7.4.5)
```

<A name="question7.4.6"></A>

```{block2, type='rmdquestion'}
6. Adjust the means in such a way that the main effect of endorser is _not_ statistically significant but the interaction effect of endorser and sex is statistically significant. [<img src="icons/2answer.png" width=115px align="right">](#answer7.4.6)
```

<A name="question7.4.7"></A>

```{block2, type='rmdquestion'}
7. Is it possible to have a statistically significant interaction effect but no statistically significant main effects? If so, adjust the scores in Figure \@ref(fig:anova-interaction) to prove your case. [<img src="icons/2answer.png" width=115px align="right">](#answer7.4.7)
```

Interpretation of moderation requires some training because we must look beyond main effects. The fact that females score on average higher than males is irrelevant to moderation but it does affect all subgroup mean scores. So the fact that the red line is above the blue line in Figure \@ref(fig:anova-interaction) is not relevant to moderation.

Moderation concerns the differences between subgroups that remain if we remove the overall differences between groups, that is, the differences that are captured by the main effects. The remaining differences between subgroup average scores provide us with a between-groups variance. In addition, the variation of outcome scores within subgroups yields a within-groups variance. Note that within-groups variance is not visible in Figure \@ref(fig:anova-interaction) because the willingness scores of the individual participants are not shown.

We can use the between-groups and within-groups variances to execute an *F* test just like the *F* test we use for main effects. The *null hypothesis* of the *F* test on an interaction effect states that the subgroups have the same population averages if we correct for the main effects. Our statistical software takes care of this correction if we include the main effects in the model, which we should always do.

Alternatively, we can formulate the null hypothesis of the test on the interaction effect as equal effects of the predictor for all moderator groups in the population. In the current example, the null hypothesis could be that the effect of endorser is the same for both sexes in the population. Or, the other way around, that the effect of sex is the same for different endorser groups in the population.

```{block2, type='rmdimportant'}
Null hypothesis of the test on the interaction effect: equal effects of the predictor for all moderator groups in the population.
```

Moderation between three or more factors is possible. These are called *higher-order interactions*. It is wise to include all main effects and lower-order interactions if we test a higher-order interaction. As a result, our model becomes very complicated and hard to interpret. If a (first-order) interaction between two predictors must be interpreted as different differences, an interaction between three factors must be interpreted as different differences in differences. That's difficult to imagine, so let us avoid them.

### Assumptions for two-way analysis of variance

The assumptions for two-way analysis of variance are the same as for one-way analysis of variance (Section \@ref(anova-assumpt)). Just note that equal group sizes and equal population variances now apply to the subgroups formed by the combination of the two factors.

### Answers {.unnumbered}

<A name="answer7.4.1"></A>

```{block2, type='rmdanswer'}
Answer to Question 1. 

* A main effect exists if the groups on one predictor (factor) have unequal
means.

![](figures/S7_7Q1.png)

* In the right-most column of the table, it is easy to see that females have a higher average willingness score than males, so there is a main effect of sex.
* In this particular plot, the red line is everywhere above the blue line, so
it is relatively easy to see that females score on average higher than males.
* If the lines cross, it will be more difficult to see a main effect. In that
case, you have to guess the average score of females over all endorsers (the
average of the red line) and the average score of all males (average of the
blue line) to see whether females score on average higher than males. [<img src="icons/2question.png" width=161px align="right">](#question7.4.1)
```

<A name="answer7.4.2"></A>

```{block2, type='rmdanswer'}
Answer to Question 2. 

* Now we have to look at the average scores for the three endorsers, which are quite clearly different (bottom row of the table). So yes, there is a main effect of endorser on willingness.

![](figures/S7_7Q1a.png)

* In the plot, we have to guess the average score for an endorser, which is
somewhere in the middle of the green arrow linking the score of females to the score of males. The average score with Clooney as endorser seems to be higher than the average score without a celebrity endorser, and the average score for Angelina Jolie is highest. These differences indicate a main effect for endorser. [<img src="icons/2question.png" width=161px align="right">](#question7.4.2)
```

<A name="answer7.4.3"></A>

```{block2, type='rmdanswer'}
Answer to Question 3. 

* The difference between average willingness to donate for females and average willingness to donate for males expresses the effect of sex. The green arrows represent these differences.
* The green arrows are equally long for participants who saw Clooney, Jolie, or no celebrity endorser. Therefore, the effect of sex on willingness to donate is the same for all three types of endorser.
* Note: If the effects of one variable (here: sex) are the same for all groups on another variable (here: endorser), there is no moderation. [<img src="icons/2question.png" width=161px align="right">](#question7.4.3)
```

<A name="answer7.4.4"></A>

```{block2, type='rmdanswer'}
Answer to Question 4. 

* Change the means for the males or females such that the green arrows have different lengths. Now, the red and dashed blue lines are no longer parallel. 
* Sex has different effects for different endorsers: Different differences! We say that sex moderates the effect of endorser on willingness to donate or that endorser moderates the effect of sex. [<img src="icons/2question.png" width=161px align="right">](#question7.4.4)
```

<A name="answer7.4.5"></A>

```{block2, type='rmdanswer'}
Answer to Question 5. 

* If the app loads, there is a main effect of endorser. The effect size
expressed by eta2 is clearly larger than zero and the _F_ test on the endorser
main effect is highly significant (p < .001).
* The main effect indicates that the endorser group averages (bottom row of the
table) are (clearly) different. We should maintain these differences if we
want to retain the main effect.
* The interaction effect is initially zero and not statistically significant.
This means that the differences between females and males (expressed by the
green arrows) do not vary (sufficiently) between the endorsers. In the initial
situation, the difference between females and males is exactly the same for
all endorsers, namely 1.5.
* Change the difference between males and females within one or more endorser
categories to create a sizable and statistically significant interaction
effect.

![](figures/S7_4Q5.png)

* But ensure that the total differences between the endorsers remain. For
example, increase the female scores for one endorser and lower the male scores
for that endorser such that the average for the endorser remains the same. [<img src="icons/2question.png" width=161px align="right">](#question7.4.5)
```

<A name="answer7.4.6"></A>

```{block2, type='rmdanswer'}
Answer to Question 6. 

* To obtain a non-significant main effect of endorser, the mean scores for the
endorsers (bottom row of the table) must be more or less the same. So lower
the scores for females and males for endorsers with high average scores and
increase the scores for endorsers with low scores until the means in the
bottom row are equal.
* If you want to have an interaction effect, you must ensure that the
difference between females and males is clearly different for different
endorsers. For example, increase the male scores for one endorser and decrease
the female scores with the same amount. [<img src="icons/2question.png" width=161px align="right">](#question7.4.6)
```

<A name="answer7.4.7"></A>

```{block2, type='rmdanswer'}
Answer to Question 7. 

* Yes, it is possible to have a statistically significant interaction effect but no statistically significant main effects.
* Adjust the scores in the table such that both the row totals (average per endorser) and the column totals (average per sex) are equal. The easiest approach is to start with the same value in all cells, for example, 5.
* A table with equal average scores for all subgroups does not have main effects.
* You can obtain an interaction effect in this situation by changing the
average scores for females and males for one endorser, such that the endorser average score remains the same and applying the opposite change to females and males for another endorser.

![](figures/S7_4Q7.png)

* You will see that the red and blue lines cross. [<img src="icons/2question.png" width=161px align="right">](#question7.4.7)
```

## Reporting Two-Way Analysis of Variance

The main purpose of reporting a two-way analysis of variance is to show the reader the differences between average group scores on the dependent variable between groups on the same factor (main effects) and different differences for groups on a second factor (interaction effect). A means plot is very suitable for this purpose. Conventionally, we place the predictor groups on the horizontal axis and we draw different lines for the moderator groups. But you can switch them if this produces a more appealing graph.

```{r 2way-meansplot, echo=FALSE, message=FALSE, warning=FALSE, out.width="70%", fig.asp=0.6, fig.pos='H', fig.align='center', fig.cap="An example of a means plot."}
# Basic colours and layout.
source("../apps/plottheme/styling.R")
# Create effect sizes.
donors <- haven::read_spss("data/donors.sav")
donors$sex <- factor(donors$sex, labels = names(attributes(donors$sex)$labels))
donors$endorser <- factor(donors$endorser, labels = names(attributes(donors$endorser)$labels))
library(dplyr)
d <- donors %>% group_by(sex, endorser) %>%
  summarise(mean_willing = mean(willing_post), .groups = "drop_last")
library(ggplot2)
ggplot(d, aes(x = endorser, y = mean_willing, group = sex, color = sex)) + 
  geom_path(size = 1) + 
  geom_point(size = 3) + 
  theme_general() + 
  xlab("Endorser") + 
  ylab("Average willingnss to donate") + 
  scale_y_continuous(limits = c(1, 10), breaks = seq(1, 10)) + 
  scale_color_manual(values=c(brewercolors[[1]], brewercolors[[5]]))
# Cleanup.
rm(d)
```

For the statistically informed reader, you should include the following information somewhere in your report:

-   That you used analysis of variance and the analysis of variance type (one-way or two-way).

-   The test result for every effect, consisting of the test name (*F*), the degrees of freedom, and the significance (p value). APA prescribes the following format if you report the test result within your text: *F* (*df1*, *df2*) = F value, *p* = p value. Note that *df1* is the degrees of freedom of the factor (between-groups) and *df2* is the degrees of freedom of the error (within-groups).

-   For each effect report eta-squared (eta^2^) and interpret it in terms of effect size. If you have to calculate eta-squared by hand, divide the between-groups sum of squares of an effect by the total sum of squares (SPSS: corrected total). If SPSS calculates eta-squared, also report the confidence interval for eta-squared.

-   For each effect worth interpretation, clarify which group or subgroup scores higher. Report the group means and their standard deviations or the mean difference with its confidence interval and *p* value (from the post-hoc tests) here. Note that the SPSS menu only supplies post-hoc tests for main effects of factors with more than two levels (groups).

-   Pay special attention to an interaction effect. Explain how an effect (differences between groups) of the predictor differs across groups on the moderator. This results in sentences containing three variables. For example: "Clooney as endorser increases willingness to donate more among women than among men." Do you recognize the three variables (predictor, moderator, and dependent variable) here?

-   As always, don't forget to mention the units (cases) and the meaning of the variables (factors and outcome). They describe the topic of the analysis.

-   Report it if the main assumption is violated, that is, if you have (sub)groups of unequal size and the test on homogeneous variances is statistically significant. Report Levene's test just like you report the *F* test of a main effect (see above). If the assumption is violated, we still report and interpret the results of the analysis of variance but we warn that the results may not be trustworthy.

A two-way analysis of variance may produce many numeric results to report. It is recommended to present them as a table (in the text or in an appendix). If you report the table, include the error, the sums of squares and mean squares in the same way that SPSS reports them. Table \@ref(tab:2way-table) presents an example.

```{r 2way-table, warning=FALSE, message=FALSE, echo=FALSE}
results <- anova(lm(willing_post ~ sex*endorser, data = donors))
results <- cbind(round(results[,2], digits = 2), round(results[,1], digits = 0), round(results[,3:4], digits = 2), round(results[,5], digits = 3))
row.names(results)[3] <- "endorser*sex"
row.names(results)[4] <- "error"
results[5,] <- c(round(sum(results[,1]), digits = 2), round(sum(results[,2]), digits = 2), NA, NA, NA)
row.names(results)[5] <- "Total"
results$`round(results[, 5], digits = 3)`[results$`round(results[, 5], digits = 3)` < 0.001] <- "< 0.001"
options(knitr.kable.NA = '')
knitr::kable(results, booktabs = TRUE, caption = "An example of a table summarizing results of a two-way analysis of variance.", col.names = c("Sum of Squares", "df", "Mean Square", "F", "p"), align = rep("r", 5))  %>%
  kable_styling(font_size = 12, full_width = F,
                latex_options = c("scale_down", "HOLD_position"))
# Cleanup.
rm(results, donors)
```

## Two-Way Analysis of Variance in SPSS {#twowaySPSS}

### Instructions

```{r SPSS2way, echo=FALSE, out.width="640px", fig.pos='H', fig.align='center', fig.cap="(ref:2waySPSS)", dev="png", screenshot.opts = list(delay = 5)}
knitr::include_url("https://www.youtube.com/embed/BUqAm1H7FW8", height = "360px")
# Video to executure two-way analysis of variance in SPSS
# Goal: Test for moderation with categorical predictors (and a numerical dependent variable) 
# Example: donors.sav, does the effect of celebrity endorsement on adults' willingness to donate differ for females and males?
# Technique: 2-way analysis of variance
# SPSS menu: GENERAL LINEAR MODEL > UNIVARIATE ; variables under Dependent Variable and Fixed Factor(s) ; PLOTS predictor under ‘Horizontal Axis’ and  under ‘Separate lines’ ; POST HOC Bonferroni ; OPTIONS ‘Descriptive Statistics’ and 'Homogeneity tests'. 
# Paste & Run.
# Check assumptions: F test homogeneous population variances or groups of equal size ; post-hoc t tests: each group more than 30 observations or normally distributed.
# Interpret output: F tests for main effects and interaction effect, statistical significance, effect sizes: manual calculation of eta2, plot for interaction effect.
```

------------------------------------------------------------------------

```{r SPSSeta2, echo=FALSE, out.width="640px", fig.pos='H', fig.align='center', fig.cap="(ref:eta2SPSS)", dev="png", screenshot.opts = list(delay = 5)}
knitr::include_url("https://www.youtube.com/embed/NWn8ty5BnDE", height = "360px")
```

### Exercises

<A name="question7.6.1"></A>

```{block2, type='rmdquestion'}
1. Use the data in [donors.sav](http://82.196.4.233:3838/data/donors.sav) to test if the effect of celebrity endorsement on adults' willingness to donate after seeing the campaign (variable *willing_post*) is different for females than for males. Check the assumptions and interpret the results. Create a plot to communicate your results. [<img src="icons/2answer.png" width=115px align="right">](#answer7.6.1)
```

<A name="question7.6.2"></A>

```{block2, type='rmdquestion'}
2. Use the same data as in Exercise 1 to test if the effect of celebrity endorsement on willingness to donate depends on remembering the campaign. Check the assumptions and interpret the results. Again, create a plot to communicate your results. [<img src="icons/2answer.png" width=115px align="right">](#answer7.6.2)
```

<A name="question7.6.3"></A>

```{block2, type='rmdquestion'}
3. Data set [smokers.sav](http://82.196.4.233:3838/data/smokers.sav) contains information about smoking for a random sample of adults. Does the attitude towards smoking depend on the adult's smoking behaviour (smoker, former smoker, or non-smoker) and on exposure to an anti-smoking campaign? 
  Recode exposure scores into three groups: low exposure (scores 3 or less), medium exposure (scores above 3 up to and including 7), and high exposure (scores above 7). [<img src="icons/2answer.png" width=115px align="right">](#answer7.6.3)
```

### Answers {.unnumbered}

<A name="answer7.6.1"></A>

```{block2, type='rmdanswer'}
Answer to Exercise 1. 

SPSS syntax:  
  
\* Check data.  
FREQUENCIES VARIABLES=willing_post endorser sex  
  /ORDER=ANALYSIS.  
\* Two-way analysis of variance.  
UNIANOVA willing_post BY endorser sex  
  /METHOD=SSTYPE(3)  
  /INTERCEPT=INCLUDE  
  /POSTHOC=endorser(BONFERRONI)   
  /PLOT=PROFILE(endorser\*sex)  
  /PRINT=HOMOGENEITY DESCRIPTIVE  
  /CRITERIA=ALPHA(.05)  
  /DESIGN=endorser sex endorser\*sex.  
  
Check data:  
  
There are no impossible values on the three variables.  
  
Check assumptions:  
  
* All six subgroups are of nearly equal size and/or the Levene test on
homogeneity of variances is not statistically significant, *F* (5, 137) = 0.43,
*p* = .825, so the most important assumption is met. Note that different
versions of SPSS may give slightly different results, but the _p_ values are all
around .830. 

* The females and males could have been drawn independently and each experimental group could have been drawn independently as well: The samples are independent. Because this is usually the case, this assumption is not always explicitly checked. But do note it if the samples are not independent. 
  
Interpret the results:  
  
The question is about moderation, so let us focus our interpretation on the  
interaction effect.  
  
* Our two-way analysis of variance shows a statistically significant interaction effect of endorser and sex of the participant, *F* (2, 137) = 4.63, *p* = .011, eta^2^ = 0.05. We may conclude that the effect of the endorsing celebrity on an adult's willingness to donate is different for females and males in the population. The differences are substantial (a medium-sized effect, eta = 0.22).
* Females who saw George Clooney are more strongly willing to donate (*M* = 5.84, *SD* = 1.48) than males (*M* = 4.09, *SD* = 1.30). In contrast, there is hardly any difference in willingness to donate between males (*M* = 4.99, *SD* = 1.77) and females (*M* = 4.91, *SD* = 1.51) who see Angelina Jolie endorse the fund raiser.  
* This result suggests that females are more sensitive to George Clooney as endorser than males but the opposite does not apply to Angelina Jolie as endorser.
  
Note that eta^2^ was calculated by hand, dividing the sum of squares of the interaction effect by the sum of squares of the corrected total. Eta is the square root of eta^2^. We do not have post-hoc tests for the interaction effect, so we do not have a confidence interval for the differences between subgroups. [<img src="icons/2question.png" width=161px align="right">](#question7.6.1)
```

<A name="answer7.6.2"></A>

```{block2, type='rmdanswer'}
Answer to Exercise 2. 

SPSS syntax:  
  
\* Check data.  
FREQUENCIES VARIABLES=willing_post endorser remember  
  /ORDER=ANALYSIS.  
\* Two-way analysis of variance.  
UNIANOVA willing_post BY endorser remember  
  /METHOD=SSTYPE(3)  
  /INTERCEPT=INCLUDE  
  /POSTHOC=endorser(BONFERRONI)   
  /PLOT=PROFILE(endorser\*remember)  
  /PRINT=HOMOGENEITY DESCRIPTIVE  
  /CRITERIA=ALPHA(.05)  
  /DESIGN=endorser remember endorser\*remember.  
  
Check data:  
  
There are no impossible values on the three variables.  
  
Check assumptions:  
  
The six subggroups are not nearly of equal size but the Levene test on homogeneity of variances is not statistically significant, *F* (5, 137) = 0.06, *p* = .997, so the most important assumptions are met. Note that different versions of SPSS may give slightly different results, but the _p_ values are all around .997. 
  
Interpret the results for the interaction effect:  
  
* For adults who do not remember the campaign, the effect of seeing Angelina Jolie endorse the fund raiser seems to be relatively low (*M* = 4.40, *SD* = 1.57) in comparison to the effect for adults who do remember the campaign (*M* = 5.48, *SD* = 1.54). This can be seen in the means plot.

![](figures/S7_6Q2.png)

* However, a two-way analysis of variance tells us that the interaction effect of endorser with campaign remembrance on willingness to donate is weak and not statistically significant, *F* (2, 137) = 0.67, *p* = .514, eta^2^ = .01.  
* Both the endorsing celebrity and campaign remembrance seem to have statistically significant effects on willingness to donate, but the effect of celebrity endorser on willingness to donate is not clearly different for adults who remember the campaign and those who do not remember the campaign.
* Our sample (_N_ = 143) is not very small, so it is not very likely that the test would have been non-significant if the effect in the population is moderate or strong. 
  
Note that eta^2^ was calculated by hand, dividing the sum of squares of the interaction effect by the sum of squares of the corrected total. [<img src="icons/2question.png" width=161px align="right">](#question7.6.2)
```

<A name="answer7.6.3"></A>

```{block2, type='rmdanswer'}
Answer to Exercise 3. 

SPSS syntax:  
  
\* Check data.  
FREQUENCIES VARIABLES=status3 exposure attitude  
  /ORDER=ANALYSIS.  
\* Group exposure to anti-smoking campaign.  
RECODE exposure (Lowest thru 3=1) (3 thru 7 = 2) (ELSE=3) INTO exposure3.  
VARIABLE LABELS  exposure3 'Exposure to anti-smoking campaign'.  
EXECUTE.  
\* Define Variable Properties.  
\*exposure3.  
VALUE LABELS exposure3  
  1.00 'Low exposure'  
  2.00 'Medium exposure'  
  3.00 'High exposure'.  
EXECUTE.  
\* Two-way analysis of variance.  
UNIANOVA attitude BY status3 exposure3  
  /METHOD=SSTYPE(3)  
  /INTERCEPT=INCLUDE  
  /POSTHOC=status3 exposure3(BONFERRONI)   
  /PLOT=PROFILE(status3\*exposure3)  
  /PRINT=HOMOGENEITY DESCRIPTIVE  
  /CRITERIA=ALPHA(.05)  
  /DESIGN=status3 exposure3 status3\*exposure3.  
  
Check data:  
  
All values on the variables seem to be valid.  
  
Check assumptions:  
  
* The nine subgroups (defined by combinations of smoking status and exposure)
are definitely not of equal size. The test on homogeneity of variances is
statistically significant, *F* (8, 76) = 2.17, *p* = .039 according to SPSS V24.
SPSS V25, however, reports four versions of the test with _F_ values ranging
between 1.849 and 2.109 and _p_ values ranging from .053 to .091. None of these
is statistically significant at the .05 level. Which makes you realize again
that statistical significance is a tricky criterion.
* We are quite confident that the variance of smoking attitude in the
population is (un)equal for all subgroups.  
* We do not meet the assumptions (or we do, if we believe SPSS V25), so we
should report this as a disclaimer with our results.
  
Interpret the results:  
  
Because we are going to interpret all effects, it is recommended to present the table with between-participants effects instead of reporting all _F_ test results in the text. The interpretation can then focus on the group averages and their differences (from the post-hoc tests).

| | SS | df | MS | _F_ | _p_ | Eta-squared |
|:----|----:|----:|----:|----:|----:|----:|
| Exposure | 50.050 | 2 | 25.025 | 26.768 | <.001 | 0.21 |
| Smoking status | 58.75 | 2 | 29.38 | 31.42 | <.001 | 0.25 |
| Smoking status * Exposure | 29.991 | 4 | 7.498 | 8.020 | <.001 | 0.13 |
| Error | 71.051 | 76 | .935 |  | | |
| Total | 233.413 | 84 |  |  | | |
_Note_. _N_ = 85.

* Exposure clearly matters to smoking attitude according to our two-way analysis of variance. High exposure is associated with a more negative attitude towards smoking (*M* = -0.52, *SD* = 1.26) than medium exposure (*M* = 0.18, *SD* = 1.95). With 95% confidence, the difference ranges from as little as 0.07 to not less than 1.3 points on a scale of ten points in the population (average difference = -0.70, *p* = .024, 95% CI [-1.33, -0.07]). At medium exposure, the attitude towards smoking is more negative than the attitude of adults with low exposure (*M* = 0.99, *SD* = 1.34). Here, the difference in the population ranges from 0.2 to 1.4 points with 95% confidence (average difference = -0.81, *p* = .006, 95% CI [-1.43, -0.19]). 
* Exposure accounts for 21 per cent of the variation in attitude towards smoking.  
  
* Smoking status also matters for attitude towards smoking. It accounts for 25 per cent of the differences in smoking attitude, which is quite a lot.   
* Former smokers (*M* = -1.69, *SD* = 1.71) are significantly more negative towards smoking than smokers (*M* = 0.80, *SD* = 1.67) and non-smokers (*M* = 0.64, *SD* = 1.17). The differences with former smokers range from 1.7 to 3.3 points (on a scale with ten points) for smokers (average difference = -2.49, *p* < .001, 95% CI [-3.30, -1.68]) and from 1.7 to 3.0 points for non-smokers (average difference = -2.33, *p* < .001, 95% CI [-3.01, -1.65] in the population).
* The difference between smokers and non-smokers is not statistically significant, although smokers seem to have a more positive rather than a more negative attitude towards smoking than non-smokers (average difference = 0.16, *p* = 1.000, 95% CI [-0.49, 0.81]). Our sample is not so large (_N_ = 85), so the power of the test is probably not high.

* There is a moderate to high (eta^2^ = 0.13, eta = 0.36) and statistically significant interaction effect of smoking status with exposure on attitude towards smoking.  
* If we inspect the means plot, we see that the effect of exposure on attitude is moderated by smoking status. Former smokers with low exposure are much more positive (less negative) about smoking than we would expect. In other words, medium and high exposure to the campaign decreases the attitude of former smokers more than the attitudes of smokers or non-smokers.

![](figures/S7_6Q3.png)
  
Note that the proportions of explained variance (eta^2^) have been calculated manually. Eta is the square root of eta^2^. [<img src="icons/2question.png" width=161px align="right">](#question7.6.3)
```

## Test Your Understanding

```{r anova-overview, fig.pos='H', fig.align='center', fig.cap="How do we recognize main effects and interaction effects in a means plot and in a table of means?", echo=FALSE, screenshot.opts = list(delay = 5), dev="png", out.width="780px"}
# Use app anova-interaction. Manipulate group means in a a means plot of a 3x2 design. Create main and interaction effects and inspect effect size and statistical significance.
source("../apps/plottheme/styling.R")
knitr::include_app("http://82.196.4.233:3838/apps/anova-interaction/", height="380px")
```

<A name="question7.7.1"></A>

```{block2, type='rmdquestion'}
1. What is or are the main effects in Figure \@ref(fig:anova-overview)? Explain how you can recognize main effects both in the means plot and in the table of means. [<img src="icons/2answer.png" width=115px align="right">](#answer7.7.1)
```

<A name="question7.7.2"></A>

```{block2, type='rmdquestion'}
2. Change some group means to create substantial and statistically significant moderation of the effect of endorser (Nobody, Clooney, or Jolie) on willingness. [<img src="icons/2answer.png" width=115px align="right">](#answer7.7.2)
```

<A name="question7.7.3"></A>

```{block2, type='rmdquestion'}
3. What is the null hypothesis of the _F_ tests reported in Figure \@ref(fig:anova-overview)? [<img src="icons/2answer.png" width=115px align="right">](#answer7.7.3)
```

<A name="question7.7.4"></A>

```{block2, type='rmdquestion'}
4. How should we interpret the eta^2^ values in Figure \@ref(fig:anova-overview)? [<img src="icons/2answer.png" width=115px align="right">](#answer7.7.4)
```

### Answers {.unnumbered}

```{block2, type='rmdanswer', echo=!ch7}
Answers to the Test Your Understanding questions will be shown in the web book when the last tutor group has discussed this chapter.
```

<A name="answer7.7.1"></A>

```{block2, type='rmdanswer', echo=ch7}
Answer to Question 1. 

* There are two factors as we can see in the table: endorser (columns) and sex (rows). Each factor has a main effect, so there are two main effects in this example.

![](figures/S7_7Q1.png)

* A main effect exists if the groups on one factor have unequal means.
* In the right-most column of the table, it is easy to see that females have a
higher average willingness score (6.5) than males (5.0), so there is a main
effect of sex.
* The average scores for the three endorsers are also quite clearly different
(bottom row of the table).
* In the plot, the main effects are more difficult to see. You have to guess
the average score of females over all endorsers (the average of the red line)
and the average score of all males (average of the blue line), to see that
females score higher than males on average. Well, the fact that the red line
is consistently higher than the blue line also tells you the story.

![](figures/S7_7Q1a.png)

* In addition, we have to guess the average score for each endorser from the
plot. This is somewhere in the middle of the green arrow linking the score of
females to the score of males for an endorser. The average score with Clooney
as endorser seems to be higher than the average score without a celebrity
endorser, and the average score for Angelina Jolie is highest. These
differences indicate a main effect for endorser. [<img src="icons/2question.png" width=161px align="right">](#question7.7.1)
```

<A name="answer7.7.2"></A>

```{block2, type='rmdanswer', echo=ch7}
Answer to Question 2. 

* An interaction effect is best understood as different effects of one factor
(predictor) across different groups on another factor (moderator). In this
example, we use endorser as predictor and sex as moderator. The red line shows us the effect of the predictor (endorser) for females. The effect is a
difference in average willingness scores for different endorsers. The
differences are visualized by the lines that link two means, for example, with Jolie as endorser, willingness to donate is higher than without a celebrity endorser.

![](figures/S7_7Q2.png)

* In the initial situation, the endorser effects are equal for females (red
lines) and males (blue lines): the Jolie versus Nobody difference (in
average willingness) is the same for females and males because the red and
blue lines have the same slopes. The lines are parallel.
* This is also reflected by the fact that the difference between average
female and male willingness for the Nobody endorser (the green arc) is the
same as the difference for Jolie as endorser. The differences (beween
females and males) are the same for all endorsers, so the endorser effect is
the same for both sexes. There is no moderation.
* The green arrows represent the differences between average willingness of
males and females for one endorser. If the green arrows are not equally long
or point in opposite directions, the differences between females and males are not the same for all endorsers.
* Change the score of one or more subgroups (sex - endorser combinations) in
the table until the _p_ value of the interaction effect is below .05. Now, the interaction effect is statistically significant at the .05 level.

![](figures/S7_7Q2a.png)

* In this situation, sex moderates the effect of endorser on willingness or,
equivalently, endorser moderates the effect of sex on willingness. The
differences in willingness to donate between females and males are not the
same for all three endorsers. In analysis of variance, moderation can be
understood as different differences. [<img src="icons/2question.png" width=161px align="right">](#question7.7.2)
```

<A name="answer7.7.3"></A>

```{block2, type='rmdanswer', echo=ch7}
Answer to Question 3. 

* The null hypothesis of an _F_ test in analysis of variance is that
all groups have the same average outcome scores in the populations from which
the samples were drawn.
* For the main effect of sex, the null hypothesis is that females and males
have the same average willingness in the population
* The null hypothesis for the main effect of endorser is that participants exposed
to Clooney, Jolie, or no celebrity endorser have the same average willingness
in the population.
* The null hypothesis of an _F_ test on an interaction effect states that all
subgroups have the same population averages if we correct for the main
effects. In this example, the null hypothesis is that all combinations of sex
and endorser have the same average willingness in the population if we correct
for the main effects.
* This can also be expressed as: The differences in average population
willingness between females and males are the same for all endorsers. [<img src="icons/2question.png" width=161px align="right">](#question7.7.3)
```

<A name="answer7.7.4"></A>

```{block2, type='rmdanswer', echo=ch7}
Answer to Question 4. 

* Eta^2^ gives the proportion of variance in the dependent variable that is
predicted or explained by the factor.
* When the app is just loaded, eta^2^ for the endorser effect is 0.47. The
differences between participants' willingness to donate scores in the sample can
be predicted for 47 per cent by the endorser that they were exposed to. [<img src="icons/2question.png" width=161px align="right">](#question7.7.4)
```

## Take-Home Points

-   In analysis of variance, we test the null hypothesis that all groups have the same population means. Behind the scenes, we actually test the ratio of between-groups variance to within-groups variance.

-   The overall differences in average outcome scores between groups on one factor (independent variable) are a main effect in an analysis of variance.

-   The differences in average outcome scores between subgroups, that is, groups that combine a level on one factor (predictor) and a level on another factor (moderator), represent an interaction effect. Note that we are dealing with the differences between subgroup scores that remain after the main effects have been removed.

-   Moderation is the phenomenon that an effect is different in different contexts. The effect can be stronger or it can have a different direction. In analysis of variance, interaction effects represent moderation.

-   Eta-squared measures the size of a main or interaction effect in analysis of variance. It tells us the proportion of variance in the dependent variable that is accounted for by the effect.

-   A means plot is very helpful for interpreting and communicating results of an analysis of variance.

-   The *F* tests in analysis of variance do not tell us which groups have different average scores on the dependent variable. To this end, we use independent-samples *t* tests as post-hoc tests with a (Bonferroni) correction for capitalization on chance.

-   To apply analysis of variance, we need a numeric dependent variable that has equal population variance in each group of a factor or each subgroup in case of an interaction effect. However, equality of population variances is not important if all groups on a factor or all subgroups in an interaction are more or less of equal size (the largest count is at most 10% of the largest count larger than the smallest count.)