proper tools for pcap file analysis in ELK stack?

Question

I'm sure this is a softball for those who are familiar with the Elastic Stack, but the docs I've read havent left it super clear.

I essentially am trying to push pcap files through the ELK stack to visualize packet information using Kibana.

I am not looking to monitor this real time, but rather have the following behavior:

1. I drop a pcap into a directory, and something picks it up (FileBeat? PacketBeat -I? LogStash?)
2. Since a pcap file isn't really useful, I might need to run it through tshark to produce readable json
3. I want this information in ElasticSearch
4. Use Kibana to make pretty graphs

From what I read PacketBeat allows for the -I option to take a pcap file as an input, but doesn't that only ship that single file? I want it to watch a directory as I drop pcaps. I guess what confused me is most of the docs talk about configuring an interface device to sniff in the packetbeat.yml

Anyway ideally I was thinking it would look something like this

packetbeat(watching for pcaps, spits out json) -> logstash (filters)-> elasticsearch (indexes)-> kibana (visualizes)

Is there a way to configure packetbeat to watch a dir for pcaps rather than an interface?

Answer 1

As of March 2021, you still can't do this natively with Packetbeat.

But you can easily "outsource" the watching of a directory tree to another tool, and have it call Packetbeat. Watchman (released by Facebook) is a good choice - it will keep track of files that have been processed. Then you could do something like the following to a) watch a directory and then b) take action when files are changed/added:

watchman watch /path/to/pcaps
watchman -- trigger ~/path/to/pcaps pcaptrigger '*.pcap' -- 'packetbeat -I'