I am solving a meta-learning problem using Reptile Algorithm as used here. I have two datasets. One contains the following classes: iris, pupil, and sclera along with their annotations. Another contains classes as follows: iris, pupil, sclera, and blood vessels along with their annotations. How to combine both datasets efficiently to train my meta-learning model?
The problem is for the first dataset we don't have annotations for blood vessels, so we can't simply merge both of them and treat it as a single dataset.
How should I tackle this problem? (please point towards some reference if possible)
I am wondering if is it a good approach to consider all of the classes from both datasets as separate subtasks to train the meta-learning model.