I want to control a 3D face model (.obj, .ply, .vrm ...) in real-time using the face landmark (or face mesh) positions obtained by face landmark detection technology in python(mediapipe face mesh, dlib, opencv...), but I can't find the relevant way of implementation, is it really impossible or I miss something?
Can anyone provide some example of implementation please? I really need it.
I tried to learn about Computer Graphics about 3d models, which 3d file formats (obj, ply, ...) are good to use and how to use them (control vertices, texture mapping...) (but I still don't understand), I also learned python's blender, opengl, open3d and other packages, but I still don't know how to modify the 3D face model according to my face landmarks or face mesh.
What I want to know is, for example, some of my face landmark points move from this position to that position and then calculate how much some vertices of the 3D face model should move, how to deform the texture map.
and whether these operations can be done in real-time.