I have some old log files from nginx that I want to import into logstash / elastic search using filebeat. These are the files in particular:
-rw-r----- 1 www-data adm 5822909 Feb 9 02:48 access-stream.log-2024-02-09-06.gz
-rw-r----- 1 www-data adm 8548121 Feb 10 03:11 access-stream.log-2024-02-10-06.gz
-rw-r----- 1 www-data adm 11714066 Feb 11 01:45 access-stream.log-2024-02-11-06.gz
I have tried following the instructions in the answer here but the logs do not appear in elasticsearch.
To clear the old data, I stopped the filebeat and logstash services and then did:
POST filebeat-7.9.1-2024.02.*/_delete_by_query
{
"query": {
"range": {
"@timestamp": {
"gte": "now-2w"
}
}
}
}
in Kibana dev tools which resulted in a gateway timeout, so I deleted the indexes:
DELETE filebeat-7.9.1-2024.02.11
and the ran the delete_by_query again that removed the old logs.
I then removed the registry and ran filebeat -e -once but no logs were imported. Restarting the services also did not import the logs.
Is there a way I can get these 3 log files imported again?