Amazon EMR: Passing java system property to custom jar

Question

Does Amazon EMR allow passing a system property to a custom jar, e.g. hadoop jar -Dkey=value myjob.jar? (key, value above used during initialization of the application itself, rather then belong to hadoop's Configuration object.)

The related thread How to specify mapred configurations & java options with custom jar in CLI using Amazon's EMR? discusses ways to pass system properties to hadoop daemons only via Bootstrap Actions, which, apparently, won't allow do the same for the java entry point class.

Answer 1

If you don't want to pass the to the mappers or reducers, you can do the following in the terminal or from a script -

export HADOOP_OPTS="-Dkey=value"
hadoop jar ...

You can also put those in $HADOOP_HOME/conf/hadoop-env.sh, if you want it for every job, without explicitly defining them every time you run a job.

Hope this makes sense.