Error While running Sqoop Export through java

Question

I have a java program to run export command. While executing the export, it tries to look for the export directory in my local machine instead of Hdfs. The same program works fine for import. I have checked the file exists on hdfs.

Please help.

Below is the code I am executing:

public void executeSqoopLoad() throws UnsupportedEncodingException{
    SqoopOptions options = new SqoopOptions();

    String driver = "oracle.jdbc.driver.OracleDriver";
    //options.setDriverClassName(driver);
    options.setUsername(“user");
    options.setPassword(“pass");
    options.setConnectString("jdbc:oracle:thin:@host:1522:rptdev");

    Configuration configuration = new Configuration(false);
    Resource configResource;
    try {
        configResource = FileUtils.getFileResource("/Users/Moiz/git/jef/hadoop/hdfs-site.xml");
        configuration.addResource(configResource.getInputStream());
        configResource = FileUtils.getFileResource("/Users/Moiz/git/jef/hadoop/core-site.xml");
        configuration.addResource(configResource.getInputStream());
        FileSystem dfs = FileSystem.get(configuration); 

        String[] uriSplit = dfs.getUri().toString().split(":");
        String newUri = uriSplit[0]+":"+uriSplit[1];
        dfs.setWorkingDirectory(new Path(newUri+"/tmp"));
        System.out.println(dfs.getWorkingDirectory());

        System.out.println("Exists = " + dfs.exists(dfs.getWorkingDirectory()));

        DateTime dt =  new DateTime();
        options.setCodeOutputDir("/tmp");
        options.setClassName("SqoopLoad_"+null+dt.getYear()+dt.getMonthOfYear()+dt.getDayOfMonth()+dt.getMillisOfDay());
        options.setVerbose(true);
        // HDFS options
        options.setExportDir(dfs.getWorkingDirectory()+"/TestDirectory");

        System.out.println("Exists = " + dfs.exists(new Path(options.getExportDir())));
        options.setInputFieldsTerminatedBy('\u0005');
        options.setTableName(“SCHEMA.TEST_SQP");
        options.setNumMappers(1);
        options.setDirectMode(true);
        System.setProperty(Sqoop.SQOOP_RETHROW_PROPERTY, "rethrow");

    } catch (FileNotFoundException e) {
        e.printStackTrace();
    } catch (IOException e) {
        e.printStackTrace();
    }

    int ret = 100;
    try{
        ret = new ExportTool().run(options);
    }catch (Exception e) {
        System.out.println("Debug");
        e.printStackTrace();
    }
    System.out.println("return code "+ ret);
}

java.lang.RuntimeException: java.io.FileNotFoundException: File /tmp/TestDirectory/part-m-00000 does not exist

Answer 1

I got it to work. Had to re-import the sqoop jars through maven.