Is there a way to select a specific point at the face after detecting facial landmarks using dlib?

Question

I am using Dlib's 68 point face landmark predictor, which has 68 points that are marked on various regions of the face shown in the picture below:

I have managed to access particular points from the predicted landmarks, for example, I can select a point that is at the corner of the lip which is the 48th point in the facial landmark predictor by the following ' import cv2 import dlib from google.colab.patches import cv2_imshow

p = "path_to_shape_predictor_68_face_landmarks.dat"
img= cv2.imread('Obama.jpg')
gray=cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
detector = dlib.get_frontal_face_detector()
predictor = dlib.shape_predictor(p)
face = detector(gray)
# Get the shape using the predictor
landmarks=predictor(gray, face)

# Defining x and y coordinates of a specific point
x=landmarks.part(48).x
y=landmarks.part(48).y
# Drawing a circle
cv2.circle(img, (x, y), 6, (0, 0, 255), -1)
cv2_imshow(img)'

It results in an image with a red small circle drawn on the specified region. However; if I want to select a point that is not a part of the 68 points of the landmark's model, how can I obtain it?

This picture will elaborate it more:

The red circle indicates the point that I have accessed using the code and the blue circle shows the desired point.

Answer 1

There are several solutions I may suggest to you:

1- The easy way is to use trigonometry and geometry, for instance to calculate pupil of left eye:

pupil_x = int((abs(landmarks.part(39).x + landmarks.part(36).x)) / 2) # The midpoint of a line Segment between eye's corners in x axis
pupil_y = int((abs(landmarks.part(39).y + landmarks.part(36).y)) / 2) # The midpoint of a line Segment between eye's corners in y axis
pupil_coordination = (pupil_x, pupil_y)

Full code:

import cv2
import dlib

detector = dlib.get_frontal_face_detector()
predictor = dlib.shape_predictor("shape_predictor_81_face_landmarks.dat")

img = cv2.imread("test.jpg")
(h, w, _) = img.shape
h2 = 600
w2 = int(h2 * h / w)
img = cv2.resize(img , (h2, w2))
img = cv2.flip(img , 1)
gray = cv2.cvtColor(img , cv2.COLOR_BGR2GRAY)
faces = detector(gray)
face = faces[0]
landmarks = predictor(gray, face)

pupil_x = int((abs(landmarks.part(39).x + landmarks.part(36).x)) / 2)
pupil_y = int((abs(landmarks.part(39).y + landmarks.part(36).y)) / 2)
pupil_coordination = (pupil_x, pupil_y)

cv2.circle(img, pupil_coordination, 6, (0, 0, 255), -1)
cv2.imshow('Show', img )
cv2.waitKey(0)
cv2.destroyAllWindows()

2- Other solution is to use larger facial landmarks model, check this on: 81 Facial Landmarks Shape Predictor

3- The hard way is to retrain and customize your own shape detector: Train a face landmarking model

For landmarks index I used the following reference: Face reference points