I hope this question fits into here because it is not only about a coding problem, but also digs into some theoretical questions concerning linear mixed-effects models. Assume the linear mixed-effects model:
model1 <- lmer(RT ~ word_duration + RT_prev + trial + stem +
(1|Subject) + (1|Word), data = df_whole)
I can compute its AIC score and use it to compare the model with other models. In my case, I have another model:
model2 <- lmer(RT ~ word_duration + RT_prev + trial + form +
(1|Subject) + (1|Word), data = df_subset)
The predictions of my model3 are the minimum of the predictions of model1 and model2 = min[model1, model2]. I would like to compare model3 with model1 and I know that I could use the mean square error (MSE) for instance. However, the MSE does not take into account that model3 is a combination of two models and a difference in MSE might not justify the increased complexity. So can I compute a measurement that takes a model's complexity into account such as the AIC in order to compare the models?
Note: model1 is trained on all the data, model2 only for a subset. This is done because I assume the items in the subset might be processed differently. Thus, for some items stem and for other form is the better predictor (as discussed in the literature).