I have this bash script.
JDBC_CON=$1
USERNAME=$2
PASSWORD=$3
HIVE_DB=$4
TABLE_NAME=$5
QUERY=$6
maxlength=500
COLUMN=$7
TARGET_DIR=$8
MAPPER=$9
echo $QUERY
sqoop import -Dorg.apache.sqoop.splitter.allow_text_splitter=true \
--connect $JDBC_CON --username $USERNAME --password $PASSWORD \
--hive-import --hive-database $HIVE_DB --hive-table $TABLE_NAME \
--fetch-size 1000 --delete-target-dir --compression-codec org.apache.hadoop.io.compress.SnappyCodec \
--as-textfile --query "$QUERY" \
--split-by $COLUMN --target-dir $TARGET_DIR -m $MAPPER;
When i run this via Talend Tsystem, the varible Query gets splitted into different variables.
Sharing log till the breakage.
INFO ] 08:21:52 xxx_xxx.import_data_0_6.Import_Data- TalendJob: 'Import_Data' - Start. [statistics] connecting to socket on port 10739 [statistics] connected [INFO ] 08:21:52 aycap_dev.import_data_0_6.Import_Data- tFileInputDelimited_1 - Retrieving records from the datasource. [INFO ] 08:21:52 aycap_dev.import_data_0_6.Import_Data- tSystem_1 - Setting the parameters. [INFO ] 08:21:52 aycap_dev.import_data_0_6.Import_Data- tSystem_1 - Root directory: '/home/talenduser/dm/'. [INFO ] 08:21:52 aycap_dev.import_data_0_6.Import_Data- tSystem_1 - Executing the command. [INFO ] 08:21:52 aycap_dev.import_data_0_6.Import_Data- tSystem_1 - Command to execute: 'sh -x import_data.sh xxxxxx talend xxxxxx default tablename SELECT /* FROM tablename WHERE $CONDITIONS column1 /user/hive/tmp '. [INFO ] 08:21:52 aycap_dev.import_data_0_6.Import_Data- tSystem_1 - The command has been executed successfully. SELECT Warning: /opt/cloudera/parcels/CDH-7.1.6-1.cdh7.1.6.p0.10506313/bin/../lib/sqoop/../accumulo does not exist! Accumulo imports will fail. Please set $ACCUMULO_HOME to the root of your Accumulo installation.
- JDBC_CON=jdbc:oracle:thin:@//xxxxxxx/orcl
- USERNAME=talend
- PASSWORD=xxxxx
- HIVE_DB=default
- TABLE_NAME=tablename
- QUERY=SELECT
- COLUMN='*'
- TARGET_DIR=FROM
- MAPPER=TALEND.tablename
- echo SELECT
- sqoop import -Dorg.apache.sqoop.splitter.allow_text_splitter=true --connect xxxxxx --username talend --password xxxxx --hive-import --hive-database default--hive-table xxxxx --fetch-size 1000 --delete-target-dir --compression-codec org.apache.hadoop.io.compress.SnappyCodec --as-textfile --query SELECT --split-by /bin /boot /data /dev /etc /home /keytab /lib /lib64 /media /mnt /opt /proc /root /run /sbin /srv /sys /tmp /usr /var --target-dir FROM -m tablename
Can someone help me on this.