Scheduling a composed task

Question

I am trying to schedule the launch of composed task in SCDF, but have been unable to find a way to do this.

The closest thing that I have found is the TriggerTask source and tasklauncher sink modules. See TriggerTask

The problem with this approach is that it takes the URI of the task artifact and launches a new task definition using given properties for the task. Therefore, launching of a composed task, or even a pre-existing task definition is not possible.

Is there are reason why the tasklauncher doesn't just launch a task like you would through the shell? This way you could just create the task definition, and provide the name to the TriggerTask source. Perhaps this functionality already exists, and I have been overlooking it.

Otherwise, I could write my own sink that invokes task launch somehow, possibly through the API.

Answer 1

At the core, the tasklauncher includes the Deployer SPI. Because of that reason, you will notice a runtime specific implementation of the same. The sole responsibility of the sink application is to orchestrate the desired "task application" upon an upstream event; whether an event from a real source or a timed schedule event via triggertask. For this reason, it requires a Boot application and we expect the application coordinates to it.

If your requirement is simply to schedule a composed task and not dynamically based on an event upstream, please consider either a standalone scheduler against the defined tasks (through APIs) in SCDF or create a simple quartz-scheduler boot application. It is probably an overkill to use the triggertask for this use-case.

If you're running in PCF, there's a scheduler-service that you could leverage and likewise, on other platforms, there can be other options.