Find an image inside of a video using python

Question

I was wondering if I am going about this the right way, or if there is a way that is much more efficient.

I am trying to look for an image inside of a video, like on every single frame of the video this image might be contained somewhere inside of it (its not the full size frame, just a small one).

Currently pulling the video into pictures as such:

import cv2
vidcap = cv2.VideoCapture('My_Video.mp4')
success,image = vidcap.read()
count = 0
success = True
while success:
  success,image = vidcap.read()
  print ('Read a new frame: ', success)
  cv2.imwrite("frame%d.jpg" % count, image)     # save frame as JPEG file
  count += 1

Then looping through them all as such:

import cv2
import numpy as np
from matplotlib import pyplot as plt

img_rgb = cv2.imread('frame1.png')
img_gray = cv2.cvtColor(img_rgb, cv2.COLOR_BGR2GRAY)
template = cv2.imread('small_icon_I_am_looking_for.png',0)
w, h = template.shape[::-1]

res = cv2.matchTemplate(img_gray,template,cv2.TM_CCOEFF_NORMED)
threshold = 0.8
loc = np.where( res >= threshold)
for pt in zip(*loc[::-1]):
    cv2.rectangle(img_rgb, pt, (pt[0] + w, pt[1] + h), (0,0,255), 2)

cv2.imwrite('res.png',img_rgb)

Is there a way to perhaps skip the saving of the pictures? I am doing this across thousands of hours of video, and saving and deleting every frame I feel will use a ton of time that might not be needed. Any ideas how I can search for this without needing to save the picture each time? This is an example of what I mean, say there was a video of super mario being played, it looks for this coin:

and detects it as such:

This currently works, but just looking for a better way.

Answer 1

If I haven't misunderstood you, the below should work. On the whole your code is well written with just minimum changes needed to do what you are asking. There was also an issue with you discarding the first frame because of the structure of your while loop. A good way to avoid this is the loop and a half/while True construct:

import cv2
import numpy as np

def process_image(img_rgb, template, count):
    img_gray = cv2.cvtColor(img_rgb, cv2.COLOR_BGR2GRAY)
    
    w, h = template.shape[::-1]

    res = cv2.matchTemplate(img_gray,template,cv2.TM_CCOEFF_NORMED)
    threshold = 0.8
    loc = np.where( res >= threshold)
    for pt in zip(*loc[::-1]):
        cv2.rectangle(img_rgb, pt, (pt[0] + w, pt[1] + h), (0,0,255), 2)

    # This will write different res.png for each frame. Change this as you require
    cv2.imwrite('res{0}.png'.format(count),img_rgb)   


def main():
    vidcap = cv2.VideoCapture('My_Video.mp4')
    template = cv2.imread('small_icon_I_am_looking_for.png',0)  # open template only once
    count = 0
    while True:
      success,image = vidcap.read()
      if not success: break         # loop and a half construct is useful
      print ('Read a new frame: ', success)
      process_image(image, template, count)
      count += 1