- update
Can I split the small test set into a validation set realB-v and a test set realB-t, then I fine-tune the model and test on the test set realB-v. Then, I swap the validation set and the test set and train a new model. Can I report the average results on two trainings?
- original post
I have a pre-trained model M trained on the real dataset realA, I test it on another real dataset realB and get very poor results because realA and realB have domain gaps. Since real images in realB are difficult to acquire, I decide to generate synthetic images like realB and use these images syntheticA to fine-tune the model M.
I wonder if I still need to get a validation set? If so, the validation set should be splitted from syntheticA or realB? realB is already a very small set (300 images).
In my view, I don't think a validation set in this case is necessary. If I directly fine-tune the model and get hyperparameters according to the accuracy rate on realB, it won't cause generalization problems because the images I use for fine-tuning are all synthetic.
I'd like to hear your views. Thank you.