OpenCV & Python: quickly superimpose mask over image without overflow

Question

I'd like to superimpose a binary mask over a color image, such that where the mask is "on", the pixel value changes by an amount that I can set. The result should look like this:

I am using OpenCV 2.4 and Python 2.7.6. I have a way that works well, but is slow, and another way that is fast but has issues with overflow and underflow. Here is the result of the faster code, with overflow/underflow artifacts:

Here is my code, showing both the fast version and the slow version:

def superimpose_mask_on_image(mask, image, color_delta = [20, -20, -20], slow = False):
    # superimpose mask on image, the color change being controlled by color_delta
    # TODO: currently only works on 3-channel, 8 bit images and 1-channel, 8 bit masks

    # fast, but can't handle overflows
    if not slow:
        image[:,:,0] = image[:,:,0] + color_delta[0] * (mask[:,:,0] / 255)
        image[:,:,1] = image[:,:,1] + color_delta[1] * (mask[:,:,0] / 255)
        image[:,:,2] = image[:,:,2] + color_delta[2] * (mask[:,:,0] / 255)

    # slower, but no issues with overflows
    else:
        rows, cols = image.shape[:2]
        for row in xrange(rows):
            for col in xrange(cols):
                if mask[row, col, 0] > 0:
                    image[row, col, 0] = min(255, max(0, image[row, col, 0] + color_delta[0]))
                    image[row, col, 1] = min(255, max(0, image[row, col, 1] + color_delta[1]))
                    image[row, col, 2] = min(255, max(0, image[row, col, 2] + color_delta[2]))

    return

Is there a fast way (probably using some of numpy's functions) to get the same result my slow code currently produces?

Answer 1

There might be better ways of applying a colorizing mask to an image, but if you want to do it the way you suggest, then this simple clipping will do what you want:

import numpy as np

image[:, :, 0] = np.clip(image[:, :, 0] + color_delta[0] * (mask[:, :, 0] / 255), 0, 255)
image[:, :, 1] = np.clip(image[:, :, 1] + color_delta[1] * (mask[:, :, 0] / 255), 0, 255)
image[:, :, 2] = np.clip(image[:, :, 2] + color_delta[2] * (mask[:, :, 0] / 255), 0, 255)

The result is:

Another way would be to simply modify the hue/saturation if your goal is to apply a color to a region. For instance:

mask = np.zeros((image.shape[0], image.shape[1]), dtype=np.bool)
mask[100:200, 100:500] = True

image = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)
image[mask, 0] = 80
image[mask, 1] = 255
image = cv2.cvtColor(image, cv2.COLOR_HSV2BGR)

Answer 2

One approach using np.clip & np.einsum -

import numpy as np

# Get clipped values after broadcasted summing of image and color_delta
clipvals = np.clip(image + color_delta,0,255)

# Mask of image elements to be changed
mask1 = mask[:,:,0]>0

# Extract clipped values for TRUE values in mask1, otherwise keep image 
out = np.einsum('ijk,ij->ijk',clipvals,mask1) + np.einsum('ijk,ij->ijk',image,~mask1)

Runtime tests

In [282]: # Setup inputs
     ...: M = 1000; N = 1000
     ...: image = np.random.randint(-255,255,(M,N,3))
     ...: imagecp = image.copy()
     ...: mask = np.random.randint(0,10,(M,N,3))
     ...: color_delta = np.random.randint(-255,255,(3))
     ...: 

In [283]: def clip_einsum(image,color_delta,mask):
     ...:     clipvals = np.clip(imagecp + color_delta,0,255)
     ...:     mask1 = mask[:,:,0]>0
     ...:     return np.einsum('ijk,ij->ijk',clipvals,mask1) +
                           np.einsum('ijk,ij->ijk',image,~mask1)
     ...: 

In [284]: def org_approach(image,color_delta,mask):
     ...:     rows, cols = image.shape[:2]
     ...:     #out = image.copy()
     ...:     for row in range(rows):
     ...:         for col in range(cols):
     ...:             if mask[row, col, 0] > 0:
     ...:                 image[row, col, 0] = min(255, max(0, 
                                 image[row, col, 0] + color_delta[0]))
     ...:                 image[row, col, 1] = min(255, max(0,
                                 image[row, col, 1] + color_delta[1]))
     ...:                 image[row, col, 2] = min(255, max(0,
                                 image[row, col, 2] + color_delta[2]))
     ...:                 

In [285]: %timeit clip_einsum(image,color_delta,mask)
10 loops, best of 3: 147 ms per loop

In [286]: %timeit org_approach(image,color_delta,mask)
1 loops, best of 3: 5.95 s per loop