Wait in ForkJoin Pool ( Java )

Question

I am using Fork join pool in java for multitasking. Now i came across a situation where, for every task, I need to hit a url then wait for 10 minutes and then again hit another url to read the data. Now the problem is that for those 10 minutes my CPU is idle and not starting another tasks ( more than those defined in fork join pool).

static ForkJoinPool pool = new ForkJoinPool(10);
public static void main(String[] args){
    List<String> list = new ArrayList<>();
    for(int i=1; i<=100; i++){
        list.add("Str"+i);
    }
    final Tasker task = new Tasker(list);
    pool.invoke(task);

public class Tasker extends RecursiveAction{

    private static final long serialVersionUID = 1L;
    List<String> myList;
    public Tasker(List<String> checkersList) {
        super();
        this.myList = checkersList;
    }
    @Override
    protected void compute() {
        if(myList.size()==1){
            System.out.println(myList.get(0) + "start");
            //Date start = new Date();
            try {

                    Thread.sleep(10*60*1000);

            } catch (Exception e) {
                // TODO Auto-generated catch block
                e.printStackTrace();
            }
            System.out.println(myList.get(0) + "Finished");
        }
        else{
            List<String> temp = new ArrayList<>();
            temp.add(  myList.get( myList.size()-1 )  );
            myList.remove( myList.size()-1 );

            Tasker left = new Tasker(myList);
            Tasker right = new Tasker(temp);

            left.fork();
            right.compute();
            left.join();
        }
    }

Now What should I do so that CPU picks all the tasks and then wait parallaly for them.

Answer 1

Unfortunately, ForkJoinPool does not work well in the face of Thread.sleep(), because it designed for many short tasks that finish quickly, rather than tasks that block for a long time.

Instead, for what you are trying to accomplish, I would recommend using ScheduledThreadPoolExecutor and dividing your task into two parts.

import java.util.*;
import java.util.concurrent.*;

public class Main {
    static ScheduledThreadPoolExecutor pool = new ScheduledThreadPoolExecutor(10);
    public static void main(String[] args){
        for(int i=1; i<=100; i++){
            pool.schedule(new FirstHalf("Str"+i), 0, TimeUnit.NANOSECONDS);
        }
    }
    static class FirstHalf implements Runnable {
        String name;
        public FirstHalf(String name) {
            this.name = name;
        }
        public void run() {
            System.out.println(name + "start");
            pool.schedule(new SecondHalf(name), 10, TimeUnit.MINUTES);
        }
    }
    static class SecondHalf implements Runnable {
        String name;
        public SecondHalf(String name) {
            this.name = name;
        }
        public void run() {
            System.out.println(name + "Finished");
        }
    }
}

If Java provides a thread pool which allows releasing the underlying resources (that is, the kernel thread participating in the thread pool) during a Thread.sleep(), you should use that instead, but I currently do not know of one.

Answer 2

According to docs forkJoin basic use section tells:

if (my portion of the work is small enough) do the work directly else split my work into two pieces invoke the two pieces and wait for the results

Hopefully this meets your need if you are using forkjoin

public class Tasker extends RecursiveAction {
    static ForkJoinPool pool = new ForkJoinPool(10);
    static int threshold = 10;
    public static void main(String[] args){
        List<String> list = new ArrayList<>();
        for(int i=1; i<=100; i++){
           list.add("Str"+i);
        }
    final Tasker task = new Tasker(list);
    pool.invoke(task);
}


private static final long serialVersionUID = 1L;
List<String> myList;

public Tasker(List<String> checkersList) {
    super();
    this.myList = checkersList;
}

void computeDirectly() {
    for(String url : myList){
        System.out.println(url + " start");
    }
    //Date start = new Date();
    try {
        //keep hitting url
        while (true) {
            for(String url : myList) {
                //url hitting code here
                System.out.println(url + " hitting");
            }
            Thread.sleep(10 * 60 * 1000);
        }

    } catch (Exception e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    }
    for(String url : myList){
        System.out.println(url + " Finished");
    }
}

@Override
protected void compute() {
    if (myList.size() <= threshold) {
        computeDirectly();
        return;
    }

    //temp list have only one url
    //List<String> temp = new ArrayList<>();
    //temp.add(  myList.get( myList.size()-1 )  );
    //myList.remove( myList.size()-1 );


    //Tasker left = new Tasker(myList);
    //Tasker right = new Tasker(temp);


    //left.fork();
    //right.compute();
    //left.join();

    List<String> first = new ArrayList<>();
    List<String> second = new ArrayList<>();

    //divide list
    int len = myList.size();
    int smHalf = len / 2;//smaller half

    first = myList.subList(0, smHalf);
    second = myList.subList(smHalf + 1, len);

    invokeAll(new Tasker(first), new Tasker(second));

}

}