using splice with socket may cause starvation

Question

I'm writing a TCP proxy, using edge-triggered epoll to monitor fd, splice to transmit data. Here is the problem: How do I know the socket receive buffer is empty?

For example, if you call read(2) by asking to read a certain amount of data and read(2) returns a lower number of bytes, you can be sure of having exhausted the read I/O space for the file descriptor.

But I found that even splice(sock, 0, pfd[1], 0, 65536, SPLICE_F_NONBLOCK) < 65536 may sometimes lead to starvation.

O_NONBLOCK enabled, n > PIPE_BUF If the pipe is full, then write(2) fails, with errno set to EAGAIN. Otherwise, from 1 to n bytes may be written (i.e., a "partial write" may occur; the caller should check the return value from write(2) to see how many bytes were actually written), and these bytes may be interleaved with writes by other processes.

So I should repeat calling splice till EAGAIN? But how can I know whether the socket receive buffer is empty or the pipe buffer is full?

Answer 1

Maybe you can use getsockopt syscall with SO_ERROR, and then you will known which socket is really EAGAIN, and then use epoll to watch the read/write event of that socket.

I also have this problem when adding reverse http proxy to my web server, I deem it should work, though I'm not sure if it is the best solution.