Sporadic memory bloat using Toub's thread pool for long running tasks?

261 views Asked by At

I have read the Toub's thread pool is a good solution for longer running tasks, so I implemented it in the following code. I'm not even sure if my implementation is a good one because I seem to have sporadic memory bloat. The process runs around 50 MB most of the time then will spike to almost a GB and stay there.

The thread pool implementation is as follows (should I even be doing this?):

private void Run()
    {
        while (!_stop)
        {
            // Create new threads if we have room in the pool
            while (ManagedThreadPool.ActiveThreads < _runningMax)
            {
                ManagedThreadPool.QueueUserWorkItem(new WaitCallback(FindWork));
            }

            // Pause for a second so we don't run the CPU to death
            Thread.Sleep(1000);
        }
    }

The method FindWork looks like this:

private void FindWork(object stateInfo)
    {
        bool result = false;
        bool process = false;
        bool queueResult = false;
        Work_Work work = null;

        try
        {
            using (Queue workQueue = new Queue(_workQueue))
            {
                // Look for work on the work queue
                workQueue.Open(Queue.Mode.Consume);
                work = workQueue.ConsumeWithBlocking<Work_Work>();

                // Do some work with the message from the queue ...

                return;

The ConsumeWithBlocking method blocks if there is nothing in the queue. Then we call return to exit the thread if we successfully retrieve a message and process it.

Typically we run 10 threads with them typically in the blocking state (WaitSleepJoin). The whole point of this is to have 10 threads running at all times.

Am I going about this all wrong?

0

There are 0 answers