Machine learning approach to facial recognition

Question

First of all i'm very new to the field. maybe my question is a bit too naive of even trivial..

I'm currently trying to understand how can i go about recognizing different faces. Here is what i tried so far and the main issues with each approach:

1) Haar Cascade -> HOG -> SVM: The main issue is that the algorithm becomes very indecisive when more than 4 people are trained.. the same occurs when we change Haar Cascade for a pre-trained CNN to detect faces..

2) dlib facial landmarks -> distance between points -> SVM or Simple Neural Network Classification: This is the current approach and it behaves very well when when 4 people are trained.. when more people are trained it becomes very messy, jumping from decision to decision and never resolves to a choice.

I've read online that Triplet loss is the way to go.. but I very confused as to how id go about implementing it.. can i use the current distance vectors found using Dlib or should i scrap everything and train my own CNN?

If i can use the distance vectors how would i pass the data to the algorithm? is Triplet loss a trivial neural network only with it's loss function altered?

I've took the liberty to show exactly how the distance vectors are being calculated:

The green lines represent the distances being calculated A 33 float list is returned which is then fed to the classifier

Here is the relevant code for the classifier (Keras):

def fit_classifier(self):
    x_train, y_train = self._get_data(self.train_data_path)
    x_test, y_test = self._get_data(self.test_data_path)
    encoding_train_y = np_utils.to_categorical(y_train)
    encoding_test_y = np_utils.to_categorical(y_test)
    model = Sequential()
    model.add(Dense(10, input_dim=33, activation='relu'))
    model.add(Dense(20, activation='relu'))
    model.add(Dense(30, activation='relu'))
    model.add(Dense(40, activation='relu'))
    model.add(Dense(30, activation='relu'))
    model.add(Dense(20, activation='relu'))
    model.add(Dense(10, activation='relu'))
    model.add(Dense(max(y_train)+1, activation='softmax'))
    model.compile(loss='mse', optimizer='adam', metrics=['accuracy'])
    model.fit(x_train, encoding_train_y, epochs=100, batch_size=10)

I think this is a more theoretical question than anything else.. if someone with good experience in the field could help me out i'd be very happy!