NoSuchMethodError while using Distcp java API

844 views Asked by At

I am trying to use Distcp Java API to do copy data from one hadoop cluster to another cluster.

However I am getting the following exception:

Exception in thread "main" java.lang.NoSuchMethodError: org.apache.hadoop.util.StringUtils.toLowerCase(Ljava/lang/String;)Ljava/lang/String;
at org.apache.hadoop.tools.util.DistCpUtils.getStrategy(DistCpUtils.java:126)
at org.apache.hadoop.tools.DistCp.createJob(DistCp.java:235)
at org.apache.hadoop.tools.DistCp.createAndSubmitJob(DistCp.java:174)
at org.apache.hadoop.tools.DistCp.execute(DistCp.java:153)
at com.monitor.BackupUtil.doBackup(BackupUtil.java:72)
at com.monitor.BackupUtil.main(BackupUtil.java:45)

I am using the following code:

    public void doBackup() throws Exception {

    System.out.println("Beginning Distcp");
    DistCpOptions options = new DistCpOptions(
            new Path(prop.getProperty("sourceClusterDirectory") + "/" + prop.getProperty("tablename")
                    + "/distcp.txt"),
            new Path(prop.getProperty("targetCluster") + prop.getProperty("targetClusterDirectory")));

    System.out.println("Disctp between--->" + prop.getProperty("sourceClusterDirectory")+ "/distcp.txt" + "AND" + prop.getProperty("targetCluster")
            + prop.getProperty("targetClusterDirectory"));
    DistCp distcp = new DistCp(new Configuration(), options);
    Job job = distcp.execute();

    job.waitForCompletion(true);

    System.out.println("DistCp Completed Successfully");


}

I am using hadoop 2.7.1 and the distcp dependency is this:

<dependency>
        <groupId>org.apache.hadoop</groupId>
        <artifactId>hadoop-distcp</artifactId>
        <version>2.7.1</version>
</dependency>
0

There are 0 answers