I am in the process of fine tuning the text to image model and Dreambooth model of Stable Diffusion v2.0. However, I want to apply regularization in case the model forgets certain concepts (class: dog, etc.) during learning, but I cannot find regulation or class-related parameters in stable diffusion's train python code.
How can I do this? Should I mix it with the train data? What ratio(train: regularization) is good?
And how should we structure the data set? I'm planning to proceed locally.(or even huggingface-hub)
Thank you for your answer.
I followed https://github.com/bmaltais/kohya_ss/tree/master and applied it to the stable diffusion 1.0 base model, which was successful. But it wasn't possible for v2.0.