efficiently saving image files to disk c#

Question

We're developing a multi-sensor aquisition tool, and we're experiencing problems with saving the images (from a webcam) to the hard drive.

We are using three threads for that purpose:

• One thread that continuously gathers the captured image from the webcam.
• A timer thread that collects the last image and sends it into a file saving method.
• The timer thread calls a third thread to save, in order not to interfere with the main timer thread functionality.

This works fine for low frequencies. When we raise the FPS to around 30, we start losing images. Notice that we have multiple sensors and not just a webcam. That's why we're using this architecture and not directly saving files from the webcam thread (we need to keep everything synced)

This is the current implementation of the save method:

private void saveImageFrame(Bitmap b, ulong frameID)
    {
        string fileSavePath = _path+ "//";
        if (b != null)
        {
            Task.Factory.StartNew(() =>
            {
                lock (_lock)
                {
                    Bitmap toSave = new Bitmap(b);
                    string fileName = fileSavePath + frameID.ToString() + ".bmp";
                    toSave.Save(fileName);
                }
            });
        }
    }

We also tried without the Task thread (for saving) and without the lock. These two result in race conditions, since the saving takes more time than the timer time interval.

I'm sure there are better ways to do this both in terms of architecture and .NET functions. Any help in enhancing the performance of this would be greatly appreciated!

Answer 1

It's quite possible that your disk just isn't fast enough. A really fast disk subsystem can sustain a write speed of perhaps 100 megabytes per second, and that's if you already have the file open. You're trying to create 30 or more files per second, which in itself is a pretty big load on the system. Add the time required to write the data, and you're pushing the file system's capabilities.

That problem will get worse as the number of files in a folder increases. If you have, say, 1,000 files in the folder, things work pretty quickly. Put 10,000 files in a single folder and you'll find that it takes a long time to create a new file in that folder.

If you are bumping up against hardware performance limitations, you have several courses of action:

1. Get faster hardware. A second, very fast, dedicated drive, for example.
2. Reduce the number of files you create. Perhaps you can save multiple images in a compressed format and then write them all as a single file to disk (a zip file, for example). Or reduce the frame rate.
3. Reduce the amount of data that you write (through compression or by reducing the image size).