Create Sqoop Hive Import Job

Question

I successfully did an ad hoc import from an oracle table into Hive table using Sqoop. Now I want to create and save a Sqoop job with the exact parameters I used for the ad hoc import.

But if I say:

sqoop job --create myjob --import \ --connect jdbc:mysql://host.port/foo \ --username myUserName \ --password-file file:///myfile.pwd \ --table mytable \ --create-hive-table \ --hive-import \ --hive-database my_hive_db \ --hive-table my_hive_table \ --fields-terminated-by '|' \ --hive-drop-import-delims \ --m 1

I get an ERROR:

19/03/05 14:56:41 ERROR tool.BaseSqoopTool: Error parsing arguments for job: 19/03/05 14:56:41 ERROR tool.BaseSqoopTool: Unrecognized argument: --import 19/03/05 14:56:41 ERROR tool.BaseSqoopTool: Unrecognized argument: --connect 19/03/05 14:56:41 ERROR tool.BaseSqoopTool: Unrecognized argument: jdbc:mysql:@host:port/foo If I delete all the \ it still doesn't work.

How do I define a Sqoop job with all the necessary parameters for Hive import?

Answer 1

You are missing space between -- and import.

Try with below syntax:

sqoop job --create myjob  -- import \
--connect jdbc:mysql://host.port/foo \
--username myUserName \
--password-file file:///myfile.pwd \
--table mytable \
--create-hive-table \
--hive-import \
--hive-database my_hive_db \
--hive-table my_hive_table \
--fields-terminated-by '|' \
--hive-drop-import-delims \
--m 1 

Answer 2

Seems to be a bash escape issue. There should be no space just after the backslack "\" characters at the end of the line. You can see the extra spaces with:

cat -A sqoop_script.sh 

Answer 3

Not sure if the example you provide is wrong, but I was always using a slightly different syntax where import was not a flag inside a job.

From the documentation, these 2 syntaxes seem familiar to me:

sqoop import (generic-args) (import-args)
sqoop-import (generic-args) (import-args)