Use some other User Model instead of settings.AUTH_USER_MODEL in django oauth toolkit

Question

1. I am using Django oauth toolkit to implement Oauth in my application.
2. I am already having a user model, but i want to use some other user model which is not AUTH_USER_MODEL.
3. What i can see now is that i can only extend all the models and use my user instead of the AUTH_USER_MODEL, but i would like to know what will be the shortest and cleanest way to implement a separate user model in django oauth toolkit

Current implementation is

user = models.ForeignKey(settings.AUTH_USER_MODEL, blank=True, null=True,
                         on_delete=models.CASCADE)

I want to use a separate AUTH_USER_MODEL