Nesting a random factor in GLM(M)

Question

I want to run a GLMM in R with a random effect that is nested into one of my fixed effects.

To describe the data a little bit:

genotype has 24 levels, and I would like to nest this within origin that has 3 levels

All these genotypes come from these 3 origins, 8 genotypes from each origin.

What would be the correct syntax for this? Should I use / or : in this case? Below is an example syntax that I have.

glmmTMB(data = data, response ~ fixed A + fixed B + origin 
   +(1|origin/genotype), family = binomial)

glmmTMB(data = data, response ~ fixed A + fixed B + origin 
    +(1|origin/genotype), family = binomial)


glmmTMB(data = data, response ~ fixed A + fixed B + origin 
   +(1|origin:genotype), family = binomial)

Answer 1

Your last example (... + origin+(1|origin:genotype)) looks correct.

• origin is handled as a fixed effect (possibly because you are interested in making statistical inferences about the differences between origins, or more likely because three levels of a grouping variable is usually insufficient to fit a random effect [unless you go to a Bayesian approach with regularizing priors/integration over the uncertainty]
• origin:genotype represents the interaction of origin and genotype, or "genotype within origin".

Also:

• origin + (1|origin/genotype) would be incorrect, because (1|origin/genotype) expands to (1|origin) + (1|origin:genotype), so you would have specified origin redundantly as both a fixed effect (origin) and a random effect (1|origin)
• if your genotypes were explicitly labeled by origin (e.g. A1, A2, ..., A8, B1, B2, ..., B8, ... rather than being labeled 1-8 in each origin), then (1|origin:genotype) and (1|genotype) would be equivalent, as far as R is concerned (but you still might want to keep the formula as (1|origin:genotype) to make it more explicit/obvious when reading the code that genotypes are nested within origin