partitioning not working in hadoop

266 views Asked by At

so in my code i have partition the data in three parts but in output i m only getting the ouput that is retuned by 0th partition even if i set no of reducers to 3

my code

      public static class customPartitioner extends Partitioner<Text,Text>{
          public int getPartition(Text key, Text value, int numReduceTasks){
    String country = value.toString();
    if(numReduceTasks==0)
        return 0;
    if(key.equals(new Text("key1")) && !value.equals(new Text("valuemy")))
        return 1%numReduceTasks;
    if(value.equals(new Text("valueother")) && key.equals(new Text("key1")) )
        return 0;
    else
        return 2%numReduceTasks;
    }
}

and set no of reducers as

         job.setNumReduceTasks(3);

it is giving me the output of only 0th partition i.e., return 0

1

There are 1 answers

0
AudioBubble On BEST ANSWER

i was doing a very silly mistake ....the partitioning was working fine in my code...but i thought the output is only in part-r-00000 file i thought that it is just to reduce load that it divides the file..but in output it shows the file by combining but i was wrong the different partitions have different outputs.