Finding out distance between output of two Convolutional Neural Network (CNN) i.e Siamese Network

Question

I am trying to build a simple Siamese neural network for usage in Human re-identification.

For that, I have used MTCNN (https://github.com/timesler/facenet-pytorch) for face detection and official pytorch implementation of arcface algorithm (https://github.com/deepinsight/insightface/tree/master/recognition/arcface_torch) for CNN implementation.

I have used a pretrained model (ms1mv3_arcface_r50_fp16) trained on resnet 50 and a backbone CNN model from their repository for implementing the CNN. The CNN takes a 112x112 image and produces an array of 512x1.

So I get two arrays as a result of the network. I have tried to compare the two array using cosine similarity but It is not giving me the correct results all the time.

So, do I need to change my model parameters or do I need to use another metric for comparison?

My code: https://gist.github.com/desertSniper87/26f5f45f4cece9d0f3008e89cea94be8

Answer 1

I've tried and got fully enjoied Elasticsearch with standard dlib face vectors (128x1)... ES can store and search&compare such kind vectors super fast and accurate. I've used somthing like this to creat a ES index:

from elasticsearch import Elasticsearch

es = Elasticsearch([{'host': ELASTIC_HOST, 'port': ELASTIC_PORT }], 
                       timeout=30, retry_on_time=True, max_retries=3,
                       http_auth=(ELASTIC_NAME, ELASTIC_PSW)
                   )

mapping = {
     "mappings": {
     "properties": {
         "face_vector":{
             "type": "dense_vector",
             "dims": 128
              },
         "pic_file": {
             "type": "text"},
         "face_loc": {
             "type": "integer"}
             }
         }
     }
 es.indices.create(index="fr_idx", body=mapping)

and then

s_body = {"size": ELASTIC_MAX_RESULTS, "min_score": tolerance,
                          "query": {
                                "script_score": {
                                    "query": {
                                        "match_all": {}
                                    },
                            "script": {
                                "source": "1 / (1 + l2norm(params.query_vector, 'face_vector'))",
                                "params": {"query_vector": UNKNOWN_FACE_ENCOD}
                                        }
                                    }
                                }
                        }
res = es.search(index="fr_idx", body=s_body) #standard index
for hit in res["hits"]["hits"]:
...

to search similar vectors.