communicating between processes with shared-memory results zero-copy?

Question

I am writing a network daemon, on Linux with kernel 2.6, which has one producer process and N of consumer processes, which does not make any change on the data, and does not create any response back to the producer.

Whenever the producer process produces a data object, of which the length varies from few 10 bytes to few 10 K-bytes, it has to pass the data object into one available consumer process.

First time, I considered to use a named/unnamed PIPE. However, they would be memory-copy overhead.

1. producer's user-space buffer --copy--> kernel-space PIPE buffer
2. kernel-space PIPE buffer --copy--> consumer's user-space buffer

Since the program may work with a large-number of peers with low latency, the copy-overhead could be harmful. Thus, I decided to use POSIX shared-memory with mmap().

I am just wondering if sharing data between processes using POSIX shared-memory with mmap() does not result any memory-copy, unlike PIPE.

Also, is there any other way to share data between processes, but results zero-copy? The program will be run on Linux with a recent version of kernel and may not have to have a cross-platform ability.

I decided not to spawn/run a thread for each consumer/produce, but a process due to design issues.

Thanks for reply.

Answer 1

Shared memory in general is designed specifically to not cause copy overhead (source: http://www.boost.org/doc/libs/1_46_0/doc/html/interprocess/sharedmemorybetweenprocesses.html#interprocess.sharedmemorybetweenprocesses.sharedmemory.shared_memory_what_is).

If you're using C++, Boost::Interprocess is a great library for implementing what you're describing in a cross-platform way -- you can use their shared memory class combined with a named_upgradable_mutex. The named_upgradable_mutex class has support for giving exclusive and sharable locks on a resource, so you can easily implement your consumer-producer model with it. (source: http://www.boost.org/doc/libs/1_37_0/doc/html/boost/interprocess/named_upgradable_mutex.html#id2913393-bb )

Answer 2

Shared-memory should not introduce any copies (cache coherency excepted), and you can directly access the memory so you may be able to avoid copies in your code.

Answer 3

Yes it should be zero-copy.

However, it's also a (possibly premature) optimisation and you need to take considerable care to ensure that your processes cooperate properly with allocation / deallocation / modifying of shared memory. You'd certainly need some kind of mutex to avoid concurrent access problems.

Personally I'd use pipes until performance becomes a proper issue. If it really does, the suggestion to use Boost::Interprocess or similar library, is a reasonable one.