Intelligent Task Scheduler

1.9k views Asked by At

Is anyone aware of a tool for creating optimal schedules for tasks? I have numerous servers, running multiple databases, with various scripts that import and transform data to and from each database.

I'm currently scheduling the various cronjobs by hand, but this is error prone and difficult to account for exceptions, such as a job taking unusually long to run because of an unusually large data load. I'm considering encoding the resources and dependencies between each task, and creating a planner to search for an optimal sequence of task executions so each task is run at a time when it's least likely to interfere with any other tasks.

I've seen Drools Planner, but it's not appropriate due it's immense complexity and overhead.

2

There are 2 answers

0
Geoffrey De Smet On

It's NP complete, so if you want anything near optimal, you can't avoid a certain amount of complexity and CPU overhead.

As I see it you got 2 choices:

  • Go for a quick construction heuristic such as First Fit Decreasing: Sort the tasks on decreasing difficulty (= the number of task they collide with, ...) and in that order, assign them to the best remaining spot. This will not be near optimal, but it will be fast, simple and with low overhead.

  • Go for real-time planning.

Out of interest, what complexity in Planner scared you away?

0
Mohan Sharma On

Task Scheduling problems comes under NP-complete set. So there is not a single algorithm to produce best answer for you.

But there are near-optimal answers.

Techniques: 1) Heuristic-based algorithms - HEFT, MinMin, MaxMin etc... 2) Meta-heuristic based algorithms - Genetic Algorithm, Particle Swarm Optimization etc...

or you can invent new algorithm.

I can share simple code that will generate near-optimal schedule if you want.