Setting a proper size for rapidjson readBuffer

Question

So I've been using rapidjson in a c++ project of mine, and I've figured out how to use it for my project needs. But while cleaning up the my code I saw that I just assigned a random number for my buffer size.

char readBuffer[80000]; rapidjson::FileReadStream readStream( file, readBuffer, sizeof( readBuffer ) );

Is there a proper way to set how large the readBuffer needs to be?

Answer 1

FileReadStream reads a chunk of bytes into the user-supplied buffer for each internal iteration. By using this stream concept, it does not need to read the whole JSON file into memory.

The buffer size may affect performance but not correctness.

The "optimal" buffer size is platform and application dependent.

If the size is too small, it will incur more overheads due to increased number of fread() calls.

Often user may use program stack (as in your example) for this buffer, so it cannot be too big as well since stack size is limited. Using a big buffer on stack may be a bigger issue for some embedded systems or applications using a lot of threads.

There are always some parameters that may affect the performance. If your application really needs optimal performance, I think the best way is to do experiments. Otherwise, I think 4096 (page size of most platforms) or above is just fine.

By the way, RapidJSON is open source and this class is really simple. Just read this header file you will know how the buffer is used.

P.S. Using vector<> is not a good practice here. As vector<> needs heap allocation and here only needs a fixed size. Using program stack is cheaper.