Why does a gRPC deadline received on a server differs from the one sent by a client

Question

I am using gRPC csharp implementation for two services to talk to one another.

When making a call from a client, I am setting a Deadline. When receiving a call on a server, I am reading what the Deadline is. To my surprise, the deadline value is not the same, it differs by a fraction of a second.

gRPC docs say that a Deadline is a fixed point in time. So I would expect to receive the same value on the server, but that is not the case.

I can see it mentioned a rounding of 100ns, but in my case it is much more.

May it account for flight time? I cannot find such logic and that would violate the notion of a fixed point in time.

Is there a bug in converting between gRPC internal Timespec and c# DateTime in the gRPC source code?

Sample (pseudo) code:

// client:

var serviceClient = new ServiceClient(new Channel("address", "port", ChannelCredentials.Insecure));
var deadline = DateTime.UtcNow;
Console.WriteLine(deadline.ToString("O"));
serviceClient.SendAsync(request, new CallOptions(null, deadline));

// server:

public override async Task Send(SendRequest request, ServerCallContext context)
{
    Console.WriteLine(context.Deadline.ToString("O"));
}

Console output is:

2020-09-22T14:14:20.8359860Z for client

2020-09-22T14:14:21.2910442Z for server

Answer 1

The gRPC library is behaving as expected. While locally on a server / client, the RPC expiration time is expressed as a deadline, when sending the RPC on the wire, the timestamp is internally converted to a timeout (and then converted back to deadline once received by the peer). This is necessary to adjust for clock skew between client / server (otherwise if the time on the server is slightly off, it would mess with the expected expiry of the RPC).

See https://github.com/grpc/grpc/blob/master/doc/PROTOCOL-HTTP2.md for what is transferred on the wire (look for "Timeout").