HighSpeed FullScreen JS Facial Detection

Question

I am currently researching a front end live camera based facial detection solution. So far I have found 3 libraries that suit my needs

1.https://github.com/auduno/clmtrackr

2.https://www.beyond-reality-face.com/overview

3.https://trackingjs.com/

All 3 look like possible solutions but tracking.js and clmtrackr seem to slow down significantly at larger player sizes (I'd love as close to a full screen application as possible, even if it means I need to hard code in the screen size). Beyond reality face is great but unfortunately the price tag that is associated is a bit beyond my resources. Has anyone had an experience with full screen face tracking in JS? I'm curious if what I'm looking for isn't possible with the open source solutions out there (not necessarily feature detection).

Thanks!

Answer 1

This is a question from a few years ago but since I have heavily messed with trackingjs for a client project of mine I wanted to chime in.

I have a 2017 MBP and trackingjs flies on it without much issue at all, despite me doing some custom computations on each frame of the video.

Internally trackingjs creates a canvas and does all the recognition by rendering each frame of the video to canvas. Some people in my office have laptops that are 3+ years older than mine and we noticed the huge performance drop on their laptop. This caused me to mess around trying to improve performance, and it ended up being much easier than you would expect.

In the source code for trackingjs, find this function tracking.trackVideo_. This does that loop via requestAnimationFrame that I mentioned.

Immediately in that function you will notice this block of code:

var resizeCanvas_ = function() {
  width = element.offsetWidth;
  height = element.offsetHeight;
  canvas.width = width;
  canvas.height = height;
};

If you set the canvas height and width to something smaller than the video player size, performance will drastically increase. For me I set canvas.width = width/2; and canvas.height = height/2; and that did the trick!

Answer 2

I'm doing something similar. My first try was with tracking.js but as you said, the performance decrease(a lot!) in big sizes... just in Chrome, but I need it to work properly in Chrome because my goal is make it as an application with NW.js.

This lack of performance is the result of tracking at every frame(I think), so I tried to modify the tracking.js code, changing the tracking frequency (the function requestAnimationFrame_ inside trackVideo_ like it's shown in this web: http://creativejs.com/resources/requestanimationframe/), but I wasn't very happy with the result.

Finally I'm using a quite old library, explained here: http://liuliu.me/eyes/javascript-face-detection-explained (used by webRTC in their samples: https://webrtc.github.io/samples/src/content/getusermedia/face) and it feeds my needs, tracking faces with a setTimeout.

I'm still testing, but results are promising!