Parsing .pcap file with rdpcap() takes long time

Question

We used tftp protocol and ethernet cable to send a media file to our remote device. At same time we used tcpdump tool at out remote device.So we saved the communication to a .pcap file.

tcpdump command we used:

tcpdump -i eth0 -s 0 -w video.pcap host <HOST_IP> and udp

We wrote this simple python code to analyze .pcap file and then merge data from frames to a output file

We aim to; Compare output file's binary information and the first media file's binary information. So we can see if there is loss at the communication line or not.

Python Code:

from scapy.all import rdpcap
a = rdpcap('video.pcap')

file = b''
print ( 'Packet Count:' , len(a))

for x in range(0,len(a)):
    if (a[x].src=='<HOST>' and a[x].dst=='<DEVICE>' and a[x].proto==17 ):   #Filter
           for y in range(  4  ,  len(a[x].load  )   ):  #We take after fourth element of payload
                file = file + a[x].load[y].to_bytes(1, 'big')
                print(x , y , a[x].load[y] )
print(file)

with open("video.mp4", "wb") as binary_file:
    # Write bytes to file
    binary_file.write(file)

The code works and creates the legit output file.But it is really slow.

For example; It takes approximately 3.5 hours to generate a 45 second 1080p 30 FPS .mp4 file from pcap file.

Answer 1

You are doing multiple things that get the code to be very slow:

1. Read the whole file

a = rdpcap('video.pcap')

You're reading the whole file in one go and storing it into memory. This is pretty slow and very resource intensive. There are ways of streaming it in Scapy, that'll be more efficient.

2. Store the buffer in memory, then write it in one go. It would be more memory-efficient to write it as it gets processed.

3. Iterate over the bytes in .load instead of using slice operators

All in all, you'd get better performance with something like:

from scapy.all import PcapReader

with PcapReader('video.pcap') as pcap:
    with open("video.mp4", "wb") as binary_file:
        for pkt in pcap:  # iterate over the file. this reads sequentially
            if pkt.src == '<HOST>' and pkt.dst == '<DEVICE>' and pkt.proto == 17:
                # Your code to process the packet:
                binary_file.write(pkt.load[4:])  # write with offset of 4