How to store data from hash in file and use it for another hash

Question

I'm new in the Perl's world and I need help regarding the save data from a hash in a file and after that use it in another hash.

This is а short example of my code:

#!/usr/local/bin/perl
use FileHandle;
use File::Copy;
use Data::Dumper qw(Dumper);
use Storable;
use warnings;
use strict;

my $user_data;

$user_data->{test_user}->{1256489043}->{STATUS}     =  "RUN";
$user_data->{test_user}->{1256489043}->{MEM}        =  "51591";
$user_data->{test_user}->{1256489043}->{RUN_TIME}   =  "41410";
$user_data->{test_user}->{1256489043}->{PROJ_NAME}  =  "unkown";
$user_data->{test_user}->{1256489043}->{GROUP}      =  "default";
$user_data->{test_user}->{1256489043}->{DATE}       =  "Aug 17 05:23";

$user_data->{test_user_2}->{528562752}->{STATUS}     =  "RUN";
$user_data->{test_user_2}->{528562752}->{MEM}        =  "591";
$user_data->{test_user_2}->{528562752}->{RUN_TIME}   =  "46410";
$user_data->{test_user_2}->{528562752}->{PROJ_NAME}  =  "unkown";
$user_data->{test_user_2}->{528562752}->{GROUP}      =  "default";
$user_data->{test_user_2}->{528562752}->{DATE}       =  "Aug 17 05:23";

store (\$user_data, 'temp_jobs.txt') or die "can't store data to $!";
my $data = retrieve('temp_jobs.txt');

print "Hash 1\n";
print Dumper \$user_data;

print "Hash2\n";
print Dumper \$data;

my @new_array_id;
foreach my $user (keys %{$user_data}) {
   foreach my $job_id (keys %{$user_data->{$user}}) {
      push (@new_array_id, $job_id)
   }
}

my @old_array_id;
foreach my $user (keys %{$data}) {
   foreach my $job_id (keys %{$data->{$user}}) {
      push (@old_array_id, $job_id)
   }
}

These are outputs from Dumper prints:

$VAR1 = \{
            'test_user_2' => {
                               '528562752' => {
                                                'GROUP' => 'default',
                                                'PROJ_NAME' => 'unkown',
                                                'DATE' => 'Aug 17 05:23',
                                                'STATUS' => 'RUN',
                                                'RUN_TIME' => '46410',
                                                'MEM' => '591'
                                              }
                             },
            'test_user' => {
                             '1256489043' => {
                                               'GROUP' => 'default',
                                               'PROJ_NAME' => 'unkown',
                                               'DATE' => 'Aug 17 05:23',
                                               'STATUS' => 'RUN',
                                               'RUN_TIME' => '41410',
                                               'MEM' => '51591'
                                             }
                           }
          };

$VAR1 = \\{
              'test_user' => {
                               '1256489043' => {
                                                 'GROUP' => 'default',
                                                 'PROJ_NAME' => 'unkown',
                                                 'DATE' => 'Aug 17 05:23',
                                                 'STATUS' => 'RUN',
                                                 'RUN_TIME' => '41410',
                                                 'MEM' => '51591'
                                               }
                             },
              'test_user_2' => {
                                 '528562752' => {
                                                  'GROUP' => 'default',
                                                  'PROJ_NAME' => 'unkown',
                                                  'DATE' => 'Aug 17 05:23',
                                                  'STATUS' => 'RUN',
                                                  'RUN_TIME' => '46410',
                                                  'MEM' => '591'
                                                }
                               }
            };

The error is observed at second foreach loop:

foreach my $user (keys %{$data}) {
   foreach my $job_id (keys %{$data->{$user}}) {
      push (@old_array_id, $job_id)
   }
}

output: Not a HASH reference at report.pl`

Actually, I'm not sure about the type of the retrieved data. Could you please help with saving the data in the new hash?

Best Regards, SK 

Answer 1

Your $user_data variable contains a hash reference. A hash reference is a scalar value. I think you're getting confused by the synopsis of the documentation for Storable which contains examples like this:

store \%table, 'file';

In the example above, %table is a hash. You, therefore, need to take a reference to it in order to pass it to store(). Because what you have in $user_data is a hash reference, not a hash, you don't need to take its reference before passing it to store(). In fact, by taking its reference, you've added an extra level of indirection which breaks your code.

You say:

Actually, I'm not sure about the type of the retrieved data.

It's a reference to a hash reference. But your code is expecting just a hash reference.

The simplest fix is to replace:

store (\$user_data, 'temp_jobs.txt') ... ;

with

store ($user_data, 'temp_jobs.txt') ... ;

With that single-character deletion, your two data structures are the same and your code works as expected.

Answer 2

I like to use Yaml::XS for basic data structures, they're easier to look at later. I've included examples of YAML::XS and Storable

Hopfully this helps

#!/usr/bin/env perl

use strict;
use warnings; 
use Storable;
use YAML::XS;
use Data::Dumper;

my $struct = {
'test_user_2' => {
   '528562752' => {
                    'GROUP' => 'default',
                    'PROJ_NAME' => 'unkown',
                    'DATE' => 'Aug 17 05:23',
                    'STATUS' => 'RUN',
                    'RUN_TIME' => '46410',
                    'MEM' => '591'
                  }
 },
'test_user' => {
 '1256489043' => {
                   'GROUP' => 'default',
                   'PROJ_NAME' => 'unkown',
                   'DATE' => 'Aug 17 05:23',
                   'STATUS' => 'RUN',
                   'RUN_TIME' => '41410',
                   'MEM' => '51591'
                 }
}

};


#print Dumper($struct);

#print Dump($struct);

YAML::XS::DumpFile("store.yaml", $struct);
my $read_yml = YAML::XS::LoadFile("store.yaml");
print "yaml out\n";
print Dumper($read_yml);

store $struct, 'store.perl_str';
my $hashref = retrieve('store.perl_str');
print "store out\n";
print Dumper($hashref);

print "looping  data\n";
for my $user_key ( sort keys %{ $hashref } )
{
    my $user = $hashref->{$user_key};
    for my $id_key ( sort keys %{ $user } )
    {
        print "$user_key - $id_key\n";
        print "   " . $user->{$id_key}{"DATE"} . "\n";
        print "   " . $user->{$id_key}{"STATUS"} . "\n";
    }
}

Answer 3

As an option you can save hash into JSON file. Following code demonstrates how desired result can be achieved.

use strict;
use warnings;
use feature 'say';

use JSON;
use Data::Dumper;

my $fname = 'datafile.json';
my $user_data;

$user_data->{test_user}{1256489043}{STATUS}      =  "RUN";
$user_data->{test_user}{1256489043}{MEM}         =  "51591";
$user_data->{test_user}{1256489043}{RUN_TIME}    =  "41410";
$user_data->{test_user}{1256489043}{PROJ_NAME}   =  "unkown";
$user_data->{test_user}{1256489043}{GROUP}       =  "default";
$user_data->{test_user}{1256489043}{DATE}        =  "Aug 17 05:23";

$user_data->{test_user_2}{528562752}{STATUS}     =  "RUN";
$user_data->{test_user_2}{528562752}{MEM}        =  "591";
$user_data->{test_user_2}{528562752}{RUN_TIME}   =  "46410";
$user_data->{test_user_2}{528562752}{PROJ_NAME}  =  "unkown";
$user_data->{test_user_2}{528562752}{GROUP}      =  "default";
$user_data->{test_user_2}{528562752}{DATE}       =  "Aug 17 05:23";

my $json = to_json($user_data);

write_json($fname,$json);

my $data = read_json($fname);

say '--- Read from file -----------------';
say Dumper($data);
say '-' x 45;
say Dumper( jobs_array($user_data) );
say '-' x 45;
say Dumper( jobs_array($data) );


sub jobs_array {
    my $data = shift;
    my @array;
    
    for my $user ( keys %{$data} ) {
        for my $job_id ( keys %{$data->{$user}} ) {
            push @array, $job_id;
        }
    }
    
    return \@array;
}


sub write_json {
    my $fname = shift;
    my $data  = shift;
    
    open my $fh, '>', $fname
        or die "Couldn't open $fname";
        
    say $fh $data;
    
    close $fh;
}

sub read_json {
    my $fname = shift;
    
    open my $fh, '<', $fname
        or "Couldn't open $fname";
        
    my $data = do{ local $/; <$fh> };
    
    close $fh;
    
    my $href = from_json($data);
    
    return $href;
}

Output

--- Read from file -----------------
$VAR1 = {
          'test_user' => {
                           '1256489043' => {
                                             'RUN_TIME' => '41410',
                                             'MEM' => '51591',
                                             'PROJ_NAME' => 'unkown',
                                             'STATUS' => 'RUN',
                                             'GROUP' => 'default',
                                             'DATE' => 'Aug 17 05:23'
                                           }
                         },
          'test_user_2' => {
                             '528562752' => {
                                              'GROUP' => 'default',
                                              'STATUS' => 'RUN',
                                              'MEM' => '591',
                                              'PROJ_NAME' => 'unkown',
                                              'DATE' => 'Aug 17 05:23',
                                              'RUN_TIME' => '46410'
                                            }
                           }
        };

---------------------------------------------
$VAR1 = [
          '1256489043',
          '528562752'
        ];

---------------------------------------------
$VAR1 = [
          '1256489043',
          '528562752'
        ];