Java batch: How to programmatically determine which partition you are running?

585 views Asked by At

In a JSR-352 batch I want to use partitioning. I can define the number of partitions via configuration or implement a PartitionMapper to do that.

Then, there are the JobContext and StepContext injectables to provide context information to my processing. However, there is no PartitionContext or the like which maintains and provides details about the partition I'm running in.

Hence the question:

How do I tell each partitioned instance of a chunk which partition it is running in so that its ItemReader can read only those items which belong to that particular partition?

If I don't do that, each partition would perform the same work on the same data instead of splitting up the input data set into n distinct partitions.

I know I can store some ID in the partition plan's properties which I can then use to set another property in the step's configuration like <property name="partitionId" value="#{partitionPlan['partitionId']}" />. But this seems overly complicated and fragile because I'd have to know the name of the property from the partition plan and must remember to always set another property to this value for each step.

Isn't there another, clean, standard way to provide partition information to steps?

Or, how else should I be splitting work by partitions and assign it to different ItemReader instances in the same partitioned chunk?

Update:

It appears that jberet has the org.jberet.cdi.PartitionScoped CDI scope, but it's not part of the JSR standard.

1

There are 1 answers

0
cheng On

When defining a partition with either partition plan (XML), or partition mapper (programatical), include these information as partition properties, and then reference these partition properties within item reader/processor/writer properties.

This is the standard way to tell item reader and other batch artifacts what resource to handle, where to begin, and where to end. This is not much different from non-partitioned chunk configuration, where you also need to configure the source and range of input data with batch properties.

For example, please org.jberet.test.chunkPartitionFailComplete.xml from one of the jberet test apps.