hdp:tool-tasklet does not propagate properties

98 views Asked by At

I am using the hdp:tool-tasklet to execute a data transformation job on hadoop. Reading from the documentation, it seems one can specify a property to be available when the actual implementation is invoked - at least for the tool-runner:

<hdp:tool-runner id="someTool" tool-class="org.foo.SomeTool" run-at-startup="true">
   <hdp:arg value="data/in.txt"/>
   <hdp:arg value="data/out.txt"/>

   **property=value**
</hdp:tool-runner>

The documentation also mentions that:

The (tool) tasklet element supports the same configuration options as #hadoop:tool-runner[tool-runner] except for run-at-startup (which does not apply for a workflow)

My tasklet is defined like this:

 <hdp:tool-tasklet id="transformTask"
    jar="classpath:hadoop-artifacts/transform-job.jar"
    scope="step">
    <hdp:arg value="#{jobParameters['inputFile']}" />
    <hdp:arg value="${spring.hadoop.fsUri}/${transform.output.path}" />
    ingest.schemaXml=#{jobExecutionContext['schemaXml']}
</hdp:tool-tasklet>

I am trying to use it later in a bean definition (within the transform-job.jar) :

<bean id="schemaDecoder" class="com.foo.bar.transform.beans.SchemaDecoder">
    <constructor-arg value="${ingest.schemaXml}"></constructor-arg>
</bean>

and I get an exception:

Could not resolve placeholder 'ingest.schemaXml' in string value "${ingest.schemaXml}"

However, if I use the hdp:jar-tasklet instead to define my job, the property gets propagated. I get into other problems with classloaders though...

So my question is, are user defined properties supported by the tool-tasklet? And if not, what are my options? The value of the property is defined dynamically at run time, so property files would not work..

thank you in advance,

mihai

0

There are 0 answers