Issue with BackProjection and Opencv

Question

I am trying to implement a backprojection algorithm with opencv to detect hands. This algorithm is composed from multiple sources. I have tried multiple methods such as morphologies and adding backgroundSubtraction to the projection to try to get a better result.I looked online too. However, I keep on getting the pic below. Does anyone have suggestions on what I am possibly doing wrong?

-Thank you

Here is my code with just the backProjection:

import cv2
import numpy as np

#module for esc keyMap on my computer
import keyMappings as kM

#set up webcam
cap = cv2.VideoCapture(0)
cap.set(cv2.cv.CV_CAP_PROP_FRAME_WIDTH, 1000) 
cap.set(cv2.cv.CV_CAP_PROP_FRAME_HEIGHT, 600)

#read a picture of a hand from my desktop
Hand = cv2.imread('/home/lie/Desktop/handPic.jpg')

#convert HSV and calc Histogram of this Pic
hsvHand = cv2.cvtColor(Hand, cv2.COLOR_BGR2HSV)
h,s,v = cv2.split(hsvHand)
roihist = cv2.calcHist([hsvHand], [0,1], None, [180,256],[0,180,0,256])
cv2.normalize(roihist,roihist,0,255,cv2.NORM_MINMAX)

#while not pressing esc
while cv2.waitKey(30) != kM.esc:

  #take pic convert HSV
  _,frame = cap.read() 
  hsvt = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)

  #backproject
  dst = cv2.calcBackProject([hsvt],[0,1],roihist,[0,180,0,256],1)


  #filtering
  disc = cv2.getStructuringElement(cv2.MORPH_ELLIPSE,(5,5))
  cv2.filter2D(dst,-1,disc,dst)



  #threshold
  ret,thresh = cv2.threshold(dst,50,255,0)


   #find contours in thresholded pic
   contours, hierarchy = cv2.findContours(thresh,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)


   ci =0
   max_area =0

   if len(contours)!=0:
      #find max contour
      for i in range(len(contours)):
        cnt = contours[i]
        area = cv2.contourArea(cnt)
        if(area>max_area):
            max_area = area
            ci =i
      #create hull around contour
      cnt = contours[ci]
      hull = cv2.convexHull(cnt)

 #Code to draw contours and show pic is ommited

This is the image used to help recognize a hand:

This is the thresholded picture:

The picture obviously shows not much of the hand and a lot of noise.

Answer 1

Its a very very long time since this post, and frankly I'm a bit surprised nobody responded to you. Although I'm quite sure you've found the answer/alternative solution, for the benefit of others I'll answer this.

What you are observing is directly caused by the number of bins that you are using in your H-S histogram. A larger number of bins means that you will be more finely representing skin, meaning that your eventual histogram will not be able to build up a "trend" of skin. You should absolutely reduce the number of bins. In my experience, about anything between 8 and 12 for both Hue and Saturation channels works well.

However, doing this isn't going to guarantee a very nice backprojected image. You still have a major issue and that is you're using the entire hand template to generate your histogram. As it is, the template contains a lot of skin, but a lot of area around the hand which is not skin. The eventual histogram that you generate will represent background as well. In my experience with this (and I've got a lot of it), even relatively small amounts of noise filtering into the eventual histogram will cause large amounts of noise in your backprojected image. Keeping background in your histogram to a minimum is key. So consider rather taking a small area inside your hand template and generating your histogram only on that. I modified your code as below and it works like a dream on my config.

Note that I made a few changes e.g. this just uses the button 'q' to quit, and the path to the template has been modified, etc. It is mostly the same.

import cv2
import numpy as np


#set up webcam

cap = cv2.VideoCapture(0)
#cap.set(cv2.cv.CV_CAP_PROP_FRAME_WIDTH, 640)
#cap.set(cv2.cv.CV_CAP_PROP_FRAME_HEIGHT, 480)

#read a picture of a hand from my desktop
Hand = cv2.imread('handPic.jpg')
hh,hw, __ = Hand.shape
#convert HSV and calc Histogram of this Pic
hsvHand = cv2.cvtColor(Hand[220:292, 110:220], cv2.COLOR_BGR2HSV)
h,s,v = cv2.split(hsvHand)
roihist = cv2.calcHist([hsvHand], [0,1], None, [12,12],[0,181,0,256])
#cv2.normalize(roihist,roihist,0,255,cv2.NORM_MINMAX)

#while not pressing esc
while cv2.waitKey(30) & 0xFF != ord('q'):

  #take pic convert HSV
  _,frame = cap.read()
  hsvt = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)

  #backproject
  dst = cv2.calcBackProject([hsvt],[0,1],roihist,[0,180,0,256],1)


  #filtering

  # ret, dst = cv2.threshold(dst,0,255,cv2.THRESH_OTSU)

  disc = cv2.getStructuringElement(cv2.MORPH_ELLIPSE,(5,5))
  cv2.filter2D(dst,-1,disc,dst)


  cv2.imshow("Skin Areas", dst)