How to handle SEC_I_MESSAGE_FRAGMENT when performing a DTLS handshake via the SChannel SSPI?

Question

When performing a DTLS handshake using the SChannel SSPI in Windows 10 - for which there is no documentation - how should the application handle a SEC_I_MESSAGE_FRAGMENT result from AcceptSecurityContext (ASC) or InitializeSecurityContext (ISC)?

I understand that I am meant to send the recieved fragment to the other party and call ASC/ISC again to obtain the next fragment, but when I call ASC again, this time with an empty input SECBUFFER_TOKEN, I receive nothing in the output token buffer, and it returns SEC_I_MESSAGE_FRAGMENT - suggesting it is expecting input data.

Presumably I am not successfully indicating to ASC that I want it to give me the next fragment, so how do I do this?

I have created a standalone example that reproduces my issue in this GitHub gist: https://gist.github.com/haddoncd/381c5e9542e977ca238ff16229bd9a0e/c881132ced94995402b617f38a5c8b6f8669b637

I have also included in the gist example output from the program which shows in detail the inputs that lead to the issue: https://gist.github.com/haddoncd/381c5e9542e977ca238ff16229bd9a0e/c881132ced94995402b617f38a5c8b6f8669b637#file-example_output-txt

Answer 1

According to my test, the only WRONG thing you did is forget to RESET context.handle once initialized during handshaking.

isc_status = InitializeSecurityContextW(
    &creds,
    context.initialized ? &context.handle : nullptr,
    nullptr,
    context_reqs,
    0,
    SECURITY_NATIVE_DREP,
    isc_input_buffers,
    0,
    &context.handle, // HERE
    &out_buffer_desc,
    &context.attrs,
    &context.expiry
);

asc_status = AcceptSecurityContext(
    &creds,
    context.initialized ? &context.handle : nullptr,
    &in_buffer_desc,
    context_reqs,
    SECURITY_NATIVE_DREP,
    &context.handle, // HERE
    &out_buffer_desc,
    &context.attrs,
    &context.expiry
);

BTW, you don't need to call DeleteSecurityContext on the old CtxtHandle because it's been invalid after the call.

Answer 2

A questions regarding your implementation: I noticed you didn't specify the protocol that you're using.

As said in the SCHANNEL_CRED docs, you can do so by setting schannel_cred.grbitEnabledProtocols = SP_PROT_DTLS1_2_CLIENT (or 1.1, server etc.), and then calling AcquireCredentialsHandle. How did you verify the connection used DTLS? The docs says that "If this member is zero, Schannel selects the protocol".