Hadoop Java Secondary Sort in Descending order Fails

209 views Asked by At

I am trying to do a secondary sort in, descending order, on the output of a MR job that produces a file containing a DocumentID and the time it has been edited:

123454 22
212234 20
223434 19
....

My first Maper:

...
_key.set(split_line[1]); //set as key the Article_ID
_value.set(1); // .. 1  
...

My first Reducer:

....
for (Iterator<IntWritable> it = values.iterator(); it.hasNext();)
      sum += it.next().get(); //sum  each values    

_value.set(sum);  //set sum
context.write(key, _value);
....

My Second Mapper:

String[] splits = value.toString().trim().split("\\s+"); //split 

context.write(new LongWritable(Long.parseLong(splits[1])),new Text(splits[0]));

My Second reducer:

for (Text val : values) {
            context.write(val, key);
        }

And finally the Comparator class:

 .....
 LongWritable k1 = (LongWritable) o1;
 LongWritable k2 = (LongWritable) o2;
 int cmp = k1.compareTo(k2);

 return -1 * cmp;
 ....

By using -1 it should produce the results in descending order based on the number of edits but it returns them in ascending order... Any ideas are more than welcome.... Thanks!

0

There are 0 answers