I'm using Hue for PIG scripts on amazon EMR. I want to make a shell call to get the date in a particular timezone into a variable which I will use to define an output folder path for writing the output to. Eventually I want to use a if else fi
loop to get a particular date from a week, so the time zone will be mentioned at various places in the command.
Sample Script
ts = LOAD 's3://testbucket1/input/testdata-00000.gz' USING PigStorage('\t');
STORE ts INTO 's3://testbucket1/$OUTPUT_FOLDER' USING PigStorage('\t');
Pig parameter definition in Hue:
- This works: OUTPUT FOLDER = `/bin/date +%Y%m%d`
- This doesn't work: OUTPUT FOLDER = `TZ=America/New_York /bin/date +%Y%m%d`
Both of the commands execute perfectly in the bash shell. But the second command gives the following error:
2015-06-23 21:43:42,901 [main] INFO org.apache.pig.tools.parameters.PreprocessorContext - Executing command : TZ=America/Phoenix /bin/date +%Y%m%d 2015-06-23 21:43:42,913 [main] ERROR org.apache.pig.Main - ERROR 2999: Unexpected internal error. Error executing shell command: TZ=America/Phoenix /bin/date +%Y%m%d. Command exit with exit code of 126
From the GNU manual: If a command is found but is not executable, the return status is 126.
How do I resolve this?
Configuration details:
- AMI version:3.7.0
- Hadoop distribution:Amazon 2.4.0
- Applications:Hive 0.13.1, Pig 0.12.0, Impala 1.2.4, Hue
- Underlying shell: bash
- User: hadoop (while using Pig and while using Bash)
If you need any clarifications then please do comment on this question. I will update it as needed.
If I do:
e.g.