Why does the PyAV-encoded video have a different total time in seconds than the original file

Question

I'm taking a mp4 file, decoding it frame by frame and reencoding it. I'm using PyAV and the reencoding is running on h265. When the code is running, it seems that all frames are passed from the decoder to the encoder, but at the end I have a file with 3 seconds less. For longer videos I'll have bigger differences.

I have already checked the fps, it comes out with a very small variation, even if I use the Fraction type to avoid decimal differences.

I'd like to know if it's some parametrization error, or if this is something to do with the encoder itself like keyframes and all those sort of things. Is there a way to fix it?

import av


container = av.open('input.mp4')

# create an output container to receive the transcoded video packets
output_container = av.open('output.mp4', 'w', format='mp4')
# add a stream to output container with same fps as input and h265 codec
output_stream = output_container.add_stream(
    'libx265', rate=container.streams.video[0].average_rate)
# set the output stream size
output_stream.width = 640
output_stream.height = 480

print('Starting transcoding...')

actual_frame = 0
container_total_frames = container.streams.video[0].frames

# start decoding packets and reading frames from the container
try:
    for packet_input in container.demux():
        if packet_input.dts is None:
            break
        for frame in packet_input.decode():

            try:
                actual_frame += 1
                # convert the frame to a numpy array
                img = frame.to_ndarray(format='bgr24')

                # prepare the ndarray frame and encode it
                frame_nd = av.VideoFrame.from_ndarray(img, format='bgr24')
                packet_output = output_stream.encode(frame_nd)
                output_container.mux(packet_output)
                print('Frame: {}/{}'.format(actual_frame, container_total_frames))
            except Exception as e:
                print("Error writing frame: ", e)
                break
except Exception as e:
    print('Finished transcoding')

output_container.close()

print('Finished transcoding')