how to get the filename along with absolute path to the file, whenever a new file is created using inode in linux?

Question

I doing some experiments with my linux OS (CentOS) and I want to track all the tool logs created under the same environment, tool generates the respective logs (.log extn) for tracking these changes I wrote a perl watcher which actually monitoring the directory that I set and when the new file is created it will show at the output but This is consuming a lot of memory and CPU utilization as i have set 2sec as the sleep period.

My QUESTION "Is there any better of way doing this ?" I thought of using inode table for tracking all the changes in the system. can this solve my issue ? and if yes then could please let us know the solution towards the same ?

Answer 1

It seems that you want to monitor a directory for changes. This is a complex job, but for which there are good modules. The easiest one to recommend is probably Linux::Inotify2

This module implements an interface to the Linux 2.6.13 and later Inotify file/directory change notification system.

This seems to be along the lines of what you wanted.

Any such monitor needs additional event handling. This example uses AnyEvent.

use warnings;
use strict;
use feature 'say';

use AnyEvent;
use Linux::Inotify2;

my $dir = 'dir_to_watch';

my $inotify = Linux::Inotify2->new  or die "Can't create inotify object: $!";

$inotify->watch( $dir, IN_MODIFY | IN_CREATE, sub {
    my $e = shift;
    my $name = $e->fullname;
    say "$name modified" if $e->IN_MODIFY;    # Both show the new file
    say "$name created"  if $e->IN_CREATE;    # but see comments below
});

my $inotify_w = AnyEvent->io (
    fh => $inotify->fileno, poll => 'r', cb => sub { $inotify->poll }
);

1 while $inotify->poll;

If you only care about new files then you only need one constant above. For both types of events the $name has the name of the new file. From man inotify on my system

... the name field in the returned inotify_event structure identifies the name of the file within the directory.

The inotify_event structure is suitably represented by a Linux::Inotify2::Watcher object.

Using IN_CREATE seems to be an obvious solution for your purpose. I tested by creating two files, with two redirected echo commands separated by semi-colon on the same command line, and also by touch-ing a file. The written files are detected as separate events, and so is the touch-ed file.

Using IN_MODIFY may also work since it monitors (in $dir)

... any file system object in the watched object (always a directory), that is files, directories, symlinks, device nodes etc. ...

As for tests, both files written by echo as above are reported, as separate events. But a touch-ed file is not reported, since data didn't change (the file wasn't written to).

Which is better suited for your need depends on details. For example, a tool may open a log file as it starts, only to write to it much later. The two ways above will behave differently in that case. All this should be investigated carefully under your specific conditions.

We may think of a race condition, since while the code executes other file(s) could slip in. But the module is far better than that and it does report new changes after the handler completes. I tested by creating files while that code runs (and sleeps) and they are reported.

Some other notable frameworks for event-driven programming are POE and IO::Async.

The File::Monitor does this kind of work, too.