How to make C++ code faster: Creating .csv file from camera data using CFile

Question

A high level overview is that 'CFile file's 'file.write()' method gets called for every individual integer data value (line 9) along with line 12, where I write a comma to file.

That means that for 327,680 input data integers, there are 2*327,680 = 655,360 file.write() calls. The code is very slow because of this and as a result, the code takes 3 seconds to create one csv file. How could I improve the efficiency of my code?

Note: I cannot change any declarations of the code. I must use CFile. Also, pSrc is of type uint_16_t and is containing the data that I want to store in the .csv file. The data ranges from 0 - 3000 integer values.

1           CFile file;
2           int mWidth = 512;
3           int mHeight = 640;
4           UINT i = 0;
5           char buf[80];
6           UINT sz = mHeight * mWidth; //sz = 327,680
7           while (i < sz) {
8                  sprintf_s(buf, sizeof(buf), "%d", pSrc[i]); 
9                  file.Write(buf, strlen(buf));
10                 i++;
11                 if (i < sz)  
12                        file.Write(",", 1);
13                 if (i % mWidth == 0) 
14                        file.Write("\r\n", 2);
15  }

All values are outputted in the 640x512 .csv file containing integers representing degrees Celcius.

Answer 1

Just Figured it out! Below is the implementation that seemed to get the job done.

int sz = mHeight * mWidth;

std::string uniqueCSV = "Frame " + to_string(miCurrentCSVImage + 1) + ".csv";
std::string file = capFile + "/" + uniqueCSV;
std::ofstream out;
out.open(file);

std::string data = "";

int i = 0;
while (i < sz) {
    data += to_string(pSrc[i]);
    i++;
    if (i < sz)
        data += ",";
    if (i % mWidth == 0)
        data += "\n";
}

out << data;
out.close();
miCurrentCSVImage++;

Answer 2

how about trying this use a string of a whole line size

then at every iteration add your data to the buf and a comma(by concatenating the whole line to the buf) & when you get to

 if (i % mWidth == 0)

write the whole line to the CFile and clear you buf using

something like this

UINT sz = mHeight * mWidth; std::string line = "";
while (int i < sz) { line += std::to_string(pSrc[i])) + ','; i++;
if (i % mWidth == 0) { 
file.Write(line.c_str(), line.size()); 
file.Write("\r\n", 2); 
line = ""; } }