How to skip frames before decoing in pyav to speed up the decoding process?

Question

I have a python script that can read bytes of a long video. I want to skip some frames in each second. for example the fps of the video is 24 and I want to drop those frames that are not multiplication by 3 if the frame numbers are like this list [1,2,3,4, ...,24] , and I expect this list [3,6,9,12,15,...,24]

the problem is that, if I skip the frames before decoding in pyav, the returned frames are corrupted (when I saved them I see some noises in the saved frames) and also it gave me an error. here is my python code:

    stream_options = [{'codec': 'h264'}]
    container = av.open(video_path, stream_options=stream_options)

    video_stream = [s for s in container.streams if s.type == "video"][0]
    packet_list = []
    for id_p, packet in enumerate(container.demux(video_stream)):
        packet_list.append(packet)

    skip=3
    for id_pack, packet in enumerate(packet_list):
        if id_pack % skip==0:
            frame = packet.decode()
            if len(frame):
                frame_np = frame[0].to_ndarray(format='rgb24')
                frame_list.append(frame_np)

it gave me an error as below:

co located POCs unavailable
reference picture missing during reorder
reference picture missing during reorder
Missing reference picture, default is 65636

Is there any way to drop frames before decoding in pyav? decoding all frames takes a long time