CR+LF changing to __ in HTTP response header

Question

See updates for updated question

I am using AFNetworking's AFHTTPClient for my app's service calls. My server uses HTTP digest authentication to authenticate. The service call is returning a 401 HTTP error with WWW-Authenticate header in its response as a challenge like this:

HTTP/1.1 401 
WWW-Authenticate: Digest realm="[email protected]",
    qop="auth",
    nonce="059c37d0e654fdba9c606f35ce84741998"
Content-Type: text/html; charset=utf-8

The connection:didReceiveAuthenticationChallenge: method is never called in AFURLConnectionOperation, which is the NSURLConnectionDelegate in this situation and would then call my block set in setAuthenticationChallengeBlock:. Instead, I just get the error: Error Expected status code in (200-299), got 401 in my AFHTTPRequestOperation failure block.

As far as I understand, these are the only required fields in the WWW-Authenticate header for HTTP digest authentication. It's very similar to the response in this example on Wikipedia. I don't have any actual body of the response since I'm not displaying any HTML to the user anyway. Should this matter? I'm not using the opaque field, but this should be optional. I don't support the "auth-int" quality of protection. Other than that, I don't return a Server or Date header. Are either of these required?

What is it I'm doing wrong here?

UPDATE: I am now logging the response headers on the client side (Apple doesn't provide a nice way to simply print the plain text response that I know of). The response as shown above is as it's logged on the server. The result of NSHTTPURLResponse allHeaders is:

{
    "Alternate-Protocol" = "80:quic";
    "Cache-Control" = private;
    "Content-Type" = "text/html; charset=utf-8";
    Date = "Mon, 09 Sep 2013 20:24:00 GMT";
    Server = "Google Frontend";
    "Transfer-Encoding" = Identity;
    Vary = "Accept-Encoding";
    "Www-Authenticate" = "Digest realm=\"[email protected]\",__    qop=\"auth\",__    nonce=\"f36dc1b8abc342d5c1cbad22a533d3868c\"";
}

The Server and Date headers are actually included by the server, as well as a few others. The server is Google App Engine.

But the issue seems to be that my \r\n CR+LF is being converted to __ for some reason, which messes up the header syntax. Looking into why this is...

UPDATE 2: Using only \n results in _. The HTTP requirement is to use CR+LF+SPACE for dividing lines within a header. So now my question becomes, how do I properly include this CR+LF+SPACE in my HttpServletResponse header string so that it's encoded correctly? Where in the server response process is the encoding being changed, resulting in the _ character I'm getting?

This is the server code which is creating the response:

public static void sendUnauthorizedResponse(HttpServletResponse resp, String realm, boolean isStale) throws IOException {
    String s = ",\r\n    ";
    String header = "Digest realm=\"" + realm + "\"" + s + "qop=\"auth\"" + s + "nonce=\"" + createNonce() + "\"";
    if (isStale) {
        header += s + "stale=TRUE";
    }
    resp.setHeader("WWW-Authenticate", header);
    resp.sendError(HttpServletResponse.SC_UNAUTHORIZED);
    log.info("Sent 401 Unauthorized Response:\n" + resp.toString());
}

The last line is what logs the response as shown first above, which is formatted correctly at this point.

Both the carriage return \r 0x0D and line feed \n 0x0A are being changed to the ASCII underscore _ 0x5F.

UPDATE 3: I can confirm this is not a client issue, but must be caused by the Google App Engine server. I get the same result when posting the service call in a poster tool.

UPDATE 4: Removing the CR+LF characters and including the comma separated attributes all on one line seems to work, but according to RFC 2047, lines should not be longer than 75 characters long, which it would now be:

An 'encoded-word' may not be more than 75 characters long, including 'charset', 'encoding', 'encoded-text', and delimiters. If it is desirable to encode more text than will fit in an 'encoded-word' of 75 characters, multiple 'encoded-word's (separated by CRLF SPACE) may be used.

While there is no limit to the length of a multiple-line header field, each line of a header field that contains one or more 'encoded-word's is limited to 76 characters.

So I still need to figure out how to correctly put CR+LF characters in my header string.

UPDATE 5: So it seems the maximum characters allowed per line of an HTTP header is server dependent and likely in the thousands of bytes range, which should be sufficient for the WWW-Authenticate header. But this still doesn't answer the question as to why "\r\n" characters in my header string are converted to "__". Is this a bug in Google App Engine? Hard to imagine I'm the only person to have used CR+LF+SPACE to separate my header attributes on new lines in HTTP responses.