Storing data from device to main memory

Question

I have a device vector that I continuously modify and then want to save in an HDF5 file. Because of the size of the device vector I cannot make multiple modifications and then save them to reduce the overhead that is connected with moving the data around. Below is an example of my code. Currently, saving the data during each step increases the runtime by more than a factor of 10. What could I do to speed up my code?

#include <thrust/device_vector.h>
#include <thrust/transform.h>

struct modifier
{
    template <typename T>
    __host__ __device__
    void operator()(T data)
    {
        // Making some modification, for example:
        return data + 1;
    }
};

struct writer
{
    template <typename T>
    void operator()(T data)
    {
        // Opening my HDF5 file, the subset to write on and then write on
    }
};

int main()
{
    thrust::device_vector<int> data(1000000);
    writer write;

    int i = 0;

    while(i < 1000000)
    {
        thrust::tranform(data.begin(), data.end(), data.begin(), modifier());
        write(data); // This line slows down everything by a lot
        i++;
    }

    return 0;
}

Answer 1

You could write to a buffer instead.

IO is usually one of the slowest parts of an application. Instead of writing into a file, you could for example write into an std::vector or other suitable container, then flush this buffer for example when the writer goes out of scope (in its destructor).

This comes with the drawback that if you application crashes your data will probably be lost.

The standard library uses this approach in the implementations of std::cout and std::cerr, where the former is a buffered output and the latter is an unbuffered output.