I am doing a homework to programming in C. Bonus points are for quick writing to the file in there upload test system.
I am trying to write a number of lines each comprising three space delimited decimal integer strings and then '\n' in file. The problem is, that fprintf is too slow (their reference time is more or less 1/3 faster).
I have tried a lots of possibilities (everything is in one for loop). fprintf (too slow):
fprintf(f, "%d %d %d\n", a[i], b[i], c[i]);
converting to string and then put the string into it - even worse:
sprintf(buffer, "%d", a[i]); //or: _itoa(_itoa(a[i], buffer, 10);
fputs(buffer, f);
fputc(' ', f);
is there any quick way to write integer numbers to simple text file (.txt) (the last solution has time 220ms, reference is 140ms for you to picture the time)? I have been trying and googling as hell, but nothing is working. But if the time is this short, there has to be some way!
PS: The numbers are integers all the time, size is 4 bytes, all the time in format:
a0 b0 c0
a1 b1 c1
a2 b2 c2
a3 b3 c3
etc...
More info: When I send the solution, I send only two files: file.h and file.c. No main etc... so everything is in their optimization. The solution should be in commands/algorithm (even in the description of the problem is statement, that fprintf is too slow and we should try something else to speed things up).
Thank you for everything!
Edit: since you want the whole code, here it is:
void save(const str_t * const str, const char *name)
{
FILE* f;
int i;
if(str->cnt == 0)
return;
f = fopen(name, "w");
if(f == NULL)
return;
for(i = 0; i < str->cnt; i++)
{
fprintf(f, "%d %d %d\n", str->a[i], str->b[i], str->c[i]);
}
fclose(f);
}
You can reduce the overhead of file I/O by writing to the file in large blocks to reduce the number of individual write operations.
There may be some advantage in writing exactly 4096 bytes (or some other power of two) in a single write, but that is largely file-system dependent and the code to do that becomes a little more complicated:
You might experiment with different values for CHUNK_SIZE - larger may be optimal, or you may find that it makes little difference. I suggest at least 512 bytes.
Test results:
Using VC++ 2015, on the following platform:
With a Seagate ST1000DM003 1TB 64MB Cache SATA 6.0Gb/s Hard Drive.
The results for a single test writing 100000 lines is very variable as you might expect on a desktop system running multiple processes sharing the same hard drive, so I ran the tests 100 times each and selected the minimum time result (as can bee seen in the code below the results):
Using default "Debug" build settings (4K blocks):
Using default "Release" build settings (4K blocks):
Optimisation had a proportionally similar effect on all three implementations, the fixed size chunk write was marginally faster then the "ragged" chunk.
When 32K blocks were used the performance was only slightly higher and the difference between the fixed and ragged versions negligible:
Using default "Release" build settings (32K blocks):
Using 512 byte blocks was not measurably differnt from 4K blocks:
Using default "Release" build settings (512 byte blocks):
All the above were 32bit (x86) builds. Building 64 bit code (x64) yielded interesting results:
Using default "Release" build settings (4K blocks)- 64-bit code:
The ragged block was marginally slower (though perhaps not statistically significant), the fixed block was significantly faster as was the line-by-line write (but not enough to make it faster then any block write).
The test code (4K block version):