Nested loops with threadpool

Question

I'm trying to make two nested loops concurrent.

Let's say we have the following code:

public MyClass 
{
  public void doSomething()
  {
    ExecutorService executor = Executors.newCachedThreadPool();

    for (int i = 0; i < 1000; i++)
    {
        executor.execute(new InnerLoop(i, executor));
    }

    ...
  }
}

and

public InnerLoop extends Thread 
{
  ...

  public InnerLoop(int i, ExecutorService Executor)
  {
    ...
  }

  public void run()
  {
    for (int j = 0; j < 1000; j++)
    {
        executor.execute(new InnerLoop2(i, j));
    }

    ...
  }  
}

Is there any way to make this performant? In my opinion the second executor.execute(..) call slows it down, as the threadpool is already busy with managing threads in the first loop.

Answer 1

Executor implementations which delegate to other threads return immediately after submit or execute have either, passed the job to a new thread or queued the job.

This implies that you don’t need to think about the executor “managing threads in the first loop”. Both loops, within MyClass.doSomething() and within InnerLoop.run(), can be processed quite fast as they don’t do much work. The will just submit the new jobs and return.

The main thing you have to concern about is that you might have up to one million pending InnerLoop2 jobs, depending on the order of processing which you can’t control. With an executor as returned by Executors.newCachedThreadPool() which creates a new thread for each job when there is no idle thread, this implies that you might have up to one million threads.

Whether having a million threads is a problem is highly system and implementation dependent. There are systems having no problems with lots of threads in general, they simply distribute the cpu time over all of them, others might have an efficiency decrease down to being unusable for a high number of threads, but whether a million is already a “high number” might depend, again, on the system and hardware.

So if in doubt, it is generally a good idea to use an executor which has a maximum number of threads (you may use Executors.newFixedThreadPool or new ThreadPoolExecutor) and, for your kind of task, combine it with an unbounded queue.

Answer 2

Many people are not fond of lenghty tutorials and while @Holger answered the question at hand, the real problem seems lack of understanding the general principles.

Optimum concurrency benefits come from using all available ressources efficiently. This usually means adapting to the number of cores (ie not using more active threads than cores), making threads really independent from each other, and many other things. This can be achieved using ThreadPools, but one should understand what a thread pool does.

A thorough answer would mean repeating most of it, therefore a good start is – as always – the official Java Concurrency Tutorial.