How to write a pseudocode in Mapreduce?

7.7k views Asked by At

Suppose that, have to analyze a large amount of Webserver-Access-Logs. These Logs are text files and they enlist one access per line. The first (separated by spaces) column contains the URL of the accessed page. The aim is to create a report that lists all the URLs together with the number of hits. How to write a pseudocode for each step?

1

There are 1 answers

0
OneCricketeer On

Basically all you are doing is WordCount but with URLs.

Stripped-down "psudeocode" straight from the tutorial.

class Mapper {

    final IntWritable ONE = new IntWritable(1);

    map(LongWritable key, Text value, Context context)  {
        String[] columns = value.split(" ");
        String url = columns[0];
        context.write(url, ONE);
    }  

}

class Reducer {

    IntWritable result = new IntWritable();

    reduce(Text key, Iterable<IntWritable> values, Context context) {
        int sum = sum(values);
        result.set(sum);
        context.write(key, result);
    }
}