Number of frames in a video using PyAV

Question

I use the library PyAV because it's one of the fatest available with Python.

Here is a simple example of code I would like to use:

import av

video = av.open("My_Super_Video.mp4")

total_frames = # ????

for i, frame in enumerate(video.decode(video=0)):
    img = frame.to_image()  # PIL image

    print("Frame: %d/%d ..." % (i, total_frames))

I could obviously use other libraries to load the library, however I would prefer using PyAV if possible due to its processing speed.

Question 1: Is it possible to obtain the number of frames with PyAV ? If yes, how ?

Question 2: In the case, I would consider using another library to load and process the video frame by frame. Which library would allow me to do the above with the highest speed as possible. I know the followings, but don't know how they compare:

1. PIMS On top of PyAV, could add some interesting feature ?
2. MoviePy (limited to videos which fit in RAM), but what about perf ?
3. Imageio (probably same limitation as above), but what about perf ?
4. OpenCV (probably same limitation as above), but what about perf ?
5. Others ?

Answer 1

Old question, but only partly answered. Let me answer the second question as well.

Question 1: Is it possible to obtain the number of frames with PyAV ? If yes, how ?

import av

with av.open("My_Super_Video.mp4") as container:
    total_frames = container.streams.video[0].frames

Question 2: In the case, I would consider using another library to load and process the video frame by frame. Which library would allow me to do the above with the highest speed as possible. I know the followings, but don't know how they compare: [...]

ImageIO timings:  0.497
PyAV timings:     0.908
MoviePy timings:  0.766
OpenCV timings:   0.766
OpenCV timings:   0.569 (no conversion to RGB)

ImageIO is the fastest; hands down. OpenCV comes close (14% slower), but only if you can do your processing in BGR. If you have to work in RGB then the conversion costs you dearly (54% slower ).

That said, it is highly workload-dependent and you should always benchmark with your specific setup. In practice, the difference is often negligible compared to how much time you spend processing each frame.

Here is the benchmark code for those interested:

import cv2
import av
import imageio.v3 as iio
from moviepy.editor import VideoFileClip
from PIL import Image
from timeit import Timer


# create a test video (roughly 11 sec and sane encoding)
frames = iio.imread("imageio:cockatoo.mp4", plugin="pyav")
iio.imwrite("test_video.mp4", frames, plugin="pyav", codec="h264")


def iio_read():
    total_frames = iio.improps("test_video.mp4", plugin="pyav").shape[0]
    for idx, frame in enumerate(iio.imiter("test_video.mp4", plugin="pyav")):
        foo = Image.fromarray(frame)

        # Note: I will not print in the benchmark. This will skew the result
        # print("Frame: %d/%d ..." % (idx, total_frames))

def av_read():
    with av.open("test_video.mp4") as container:
        total_frames = container.streams.video[0].frames
        for frame in container.decode(video=0):
            foo = frame.to_image()

def moviepy_read():
    # Can not read frame_count
    for frame in VideoFileClip("test_video.mp4").iter_frames():
        foo = Image.fromarray(frame)

def cv2_read():
    cap = cv2.VideoCapture("test_video.mp4")
    
    total_frames = cap.get(cv2.CAP_PROP_FRAME_COUNT)
    success, frame = cap.read()
    idx = 0
    while success:
        frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
        foo = Image.fromarray(frame)
        success, frame = cap.read()
        idx += 1

def cv2_read2():
    cap = cv2.VideoCapture("test_video.mp4")
    
    total_frames = cap.get(cv2.CAP_PROP_FRAME_COUNT)
    success, frame = cap.read()
    idx = 0
    while success:
        foo = Image.fromarray(frame)
        success, frame = cap.read()
        idx += 1

repeats = 10
time_moviepy = min(Timer("moviepy_read()", globals=globals()).repeat(repeats, number=1))
time_cv2 = min(Timer("cv2_read()", globals=globals()).repeat(repeats, number=1))
time_cv2_no_convert = min(Timer("cv2_read2()", globals=globals()).repeat(repeats, number=1))
time_iio = min(Timer("iio_read()", globals=globals()).repeat(repeats, number=1))
time_av = min(Timer("av_read()", globals=globals()).repeat(repeats, number=1))

print(
f"""
ImageIO timings:  {time_iio:.3f}
PyAV timings:     {time_av:.3f}
MoviePy timings:  {time_moviepy:.3f}
OpenCV timings:   {time_cv2:.3f}
OpenCV timings:   {time_cv2_no_convert:.3f} (no conversion to RGB)
"""
)

Package Versions:

av==10.0.0
moviepy==1.0.3
Pillow==9.4.0
opencv-python==4.7.0.68
imageio==2.25.0

Answer 2

You can get the number of frames in stream with Stream.frames attribute.

Source: http://docs.mikeboers.com/pyav/develop/api/stream.html#av.stream.Stream

Answer 3

To get the frames of the first video stream do:

container = av.open("My_Super_Video.mp4")
total_frames = container.streams.video[0].frames