Reading UDP packets containing JPG video frames with python

99 views Asked by At

I've got a cheap drone and I'm trying to read with Python the video stream coming from it. The drone communicates via UDP protocol, and by using wireshark I see it sends jpeg frames (starts with FFD8 and finishes with FFD9). Each frame is transmitted across multiple packets, each carrying a data payload of 1472 bytes. The final packet linked to a specific frame, which contains the remaining data, has a payload of less than 1472 bytes.

example of a typical frame sent over multiple packets (showing just the payload):

packet_1: f8000104ffd8ffe0....53a521a168c504ee
packet_2: f80002043694d21a....3b53378ed5719327

packet_3: f800030495221784....b9371a6834ec5349
packet_4: f8000404a50d4ab5....b915d575ac0d661d
packet_5: f8000504ad9aeec3....a34393530ad98a36
packet_6: f800060468ac460d....7af2e9372a9a9d95
packet_7: f80007049a503248....16fa5f30d4a45731
packet_8: f80008042f978a7c....14fbafbd4c55c495
packet_9: f80009042dca65f8....ee17226029368a71
packet_a: f8000a04351eea68....4da1b8267152785f
packet_b: f8010b045053f2b4....3d0940000000ffd9 # here last payload contains 201 bytes

You can see clearly that every payload is sent with a 4 byte (or 8 hexstring characters) header that relates to the order of the packets regarding the jpg data and the frame number.

I tried to read the data in a lot of different ways but no matter what I do I get a very poor image quality that barely resembles what the drone is actually looking at. Important to say that when using the app that comes with the drone the footage is clear and with no latency.

some code I used in my last iteration:

(context: I recorded some packets coming from the drone and saved it in a pickle file as a list of hexstring representing the packet payload)

import pickle

with open("packets_data.pkl", "+br") as f:
    packet_list = pickle.load(f)


frame = ""
for i, pckt in enumerate(packet_list):
    print(f"{pckt[:16]}....{pckt[-16:]}", end="  ")
    frame += pckt[8:] # removing the 4 byte header from every packet
    if len(pckt) < 2944: # less than 1472 bytes is the last packet in the frame
        print(f"frame end. last packet len: {len(pckt)}, i: {i}")
        with open(f"frames/my_frame_{i}.jpg", "+wb") as f:
            f.write(bytes.fromhex(frame))
            frame = ""
    
    print()

The result is jpeg images that look horrible, for example: enter image description here

link to packets representing a frame: https://file.io/kM9MdfqYlfSy (this can be downloaded only once)

0

There are 0 answers