I have a web server and I need to analyze the traffic and find potential anomalies in it. My first idea was to use filebeat to take the apache web access logs and pass it to the ELK stack. The nginx web access logs are in the following form:
42.236.10.125 - - [19/Dec/2020:15:23:11 +0100] "GET /templates/jp_hotel/css/template.css HTTP/1.1" 200 10004 "http://www.almhuette-raith.at/" "Mozilla/5.0 (Linux; U; Android 8.1.0; zh-CN; EML-AL00 Build/HUAWEIEML-AL00) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/57.0.2987.108 baidu.sogo.uc.UCBrowser/11.9.4.974 UWS/2.13.1.48 Mobile Safari/537.36 AliApp(DingTalk/4.5.11) com.alibaba.android.rimet/10487439 Channel/227200 language/zh-CN" "-"
42.236.10.117 - - [19/Dec/2020:15:23:11 +0100] "GET /templates/jp_hotel/css/layout.css HTTP/1.1" 200 1801 "http://www.almhuette-raith.at/" "Mozilla/5.0 (Linux; U; Android 8.1.0; zh-CN; EML-AL00 Build/HUAWEIEML-AL00) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/57.0.2987.108 baidu.sogo.uc.UCBrowser/11.9.4.974 UWS/2.13.1.48 Mobile Safari/537.36 AliApp(DingTalk/4.5.11) com.alibaba.android.rimet/10487439 Channel/227200 language/zh-CN" "-"
However, I noticed that packetbeat also exists, but based on my research I found out that it provides a similar output:
So my question is what is the difference between the two of them and should I use both? Is there a feature that packetbeat has and it might be useful for my case?
I would use packetbeat if you are reviewing strictly network traffic and NOT logs. If you are analyzing logs, use Filebeat. You can use both tools, but make sure to understand what you're trying to have the tool ingest.