Can Matlab Computer Vision toolbox detect multiple nearby or approaching humans?

Question

We want to mount one or stereo video cameras on our product, and then detect if 1 or more humans are nearby (within a specific distance) or approaching.

Can Matlab CVST do this?

Answer 1

Yes it can. In fact it ships with an example that detects people in stereo video and measures their distances to the camera. To use this approach you would need to calibrate your cameras using the Stereo Camera Calibrator app.

Answer 2

There are 2 parts to the problem. The first part is detecting and tracking humans in video imagery. The second is determining the distance to the humans.

The detection and tracking problem is well supported in the Matlab CVST by means of detecting human faces. There are several methods of detecting faces and maintaining track on them through optical flow methods over multiple frames. There is an example in the Matlab documentation of detecting and tracking faces using the KLT optical flow algorithm at this link. Of course there would probably need to be some tailoring to your specific case.

Body detection as pointed out by @ABC is certainly another option. Discussing whether to track using a Kalman filter instead of an optical flow approach depends a lot on what other objects are in your scene, lighting, and a lot of other factors. A Kalman filter can track body kinematics better, but it would also require more tuning.

The second part of the problem is more difficult. You would need to have a stereo camera setup, find point correspondences between detected faces in frames, find the epipolar geometry of the camera setup and compute distances to faces. It is doable in Matlab but you are definitely going to need some custom development.

I am not sure how comprehensive an answer you wanted but hopefully this is a start.

Answer 3

I'm going to second @paisanco's answer. But I'll break it down for you in a few more steps.

1. Perform camera calibration. Each camera has different focal and principle point which you need to do in order to have a precise detection. You can use this toolbox for it.
2. Detection - Videos are a series of frames but in each frame you're going to want to detect the human. Matlab has a human detection function that can do multiscale detection. I don't remember if this will find humans that are partials (like waist up) but for prototyping I'd start with this.
3. Tracking. For tracking humans in videos you're going to want to run a Kalman filter. What this does is it takes the center (generally speaking) of the human and tracks where the person goes. This filter allows for smoother tracking as well as tracking partial and total occlusions. Here's an example to get you started with Kalman filter tracking in Matlab. A second example which is really awesome and has a video explanation is Student Dave's implementation with video explanation.

Hope this helps.

EDIT: I forgot to answer the second part of the question: here's the update.

A quick and dirty way to get approximate distance is to take photos of the human from 1ft, 3ft, 5ft, 10ft, etc, away. Then run your human detector and it should give you the height of the major axis and you can come up with a simple equation to get distance based on height. You could also come up with a confidence measure that this is a human based on this.