I am working on Ubuntu 18.04 with Python 2.7 and OpenCV 3.2. My application is the front-end of a video pipeline and entails extracting video frames from a webcam, possibly cropping and/or rotating them (90, 180, 270 deg), and then distributing them to one or more other pieces of code for further processing. The overall system tries to maximize efficiency at every step to e.g., improve options for adding functionality later on (compute power and bandwidth wise).

Functionally, I have the front-end working, but I want to improve its efficiency by processing JPEG frames extracted from the camera's MJPEG stream. This would allow efficient, lossless cropping and rotation in the JPEG domain, e.g. using jpegtran-cffi, and distribution of compressed frames that are smaller than the corresponding decoded ones. JPEG decoding will take place if/when/where necessary, with an overall expected gain. As an extra benefit, this approach allows efficient saving of the webcam video without loss of image quality due to decoding + re-coding.

The problem I run into is that OpenCV's VideoCapture class does not seem to allow access to the MJPEG stream:

import cv2

cam = cv2.VideoCapture()
if not cam.isOpened():
    print("Cannot open camera")
    enabled = True
    while enabled:
        enabled, frame = cam.read()
        # do stuff

Here, frame is always in component (i.e., decoded) format. I looked at using cam.grab() + cam.retrieve() instead of cam.read() with the same result (in line with the OpenCV documentation). I also tried cam.set(cv2.CAP_PROP_CONVERT_RGB, False) but that only converts the decoded video to RGB (if it is in another component format) and does not prevent decoding. BTW I verified that the camera uses the MJPEG codec (via cam.get(cv2.CAP_PROP_FOURCC)).

So my questions are: am I missing something or will this approach not work? If the latter, is there an alternative?

A final point: the application has to be able to control the webcam within its capabilities; e.g., frame size, frame rate, exposure, gain, ... This is nicely supported by cv2.VideoCapture.



Follow-up: in absence of the solution I was looking for, I added explicit JPEG encoding:

jpeg_frame = cv2.imencode('.jpg', frame, [int(cv2.IMWRITE_JPEG_QUALITY), _JPEG_QUALITY])[1]

with _JPEG_QUALITY set to 90 (out of 100). While this adds computation and reduces image quality, both in principle redundant, it allows me to experiment with trade-offs. --KvZ

0 Answers