Is there a way to differentiate validation and evaluation in AllenNLP?

Question

Sometimes we may want some minor different behaviors during validation (i.e, validation on dev set during training) and final evaluation. I am not sure whether there is a simple way to inform our model whether it's called by a train command or a evaluate command. Now what I can do is just to use self.training to distinguish them from training, but both validation and evaluation have self.training == False, which makes it not possible for me to distinguish them. I mean if it's my own pytorch framework, that's extremely easy to do, since I can write my own train and evaluate methods, but in AllenNLP they are both part of the framework code that I don't really want to modify. Is there an easy way to do it?

Answer 1

You could try overriding your Model's from_archive() method to set a flag, something like self._training = False.