SendGrid Inbound Parse japanese (encoding: shift_jis) text garbled

Question

Currently we are using SendGrid Inbound Parse to receive emails. We handle the Inbound Parse webhook request by Azure HttpTrigger function implmented in C# (.NET 6). When the received email is in UTF-8 encoding, everything's okay. However, when we tried to receive email in shift_jis encoding, headers are okay, but japanese characters in text and html are garbled.

From Inbound Parse request, we got the charsets as below:

• subject: UTF-8
• to: UTF-8
• from: UTF-8
• cc: UTF-8
• html: shift_jis
• text: shift_jis

And the string we got directly from request.form["text"] (or "html") was already garbled like "�e�L�X�gshiftJis-007" (should be "テキストshiftJis-007"), so we cannot use string in request directly.

Then we tried to convert (System.Text.Encoding.Convert method) it from charset encoding (shift_jis) to utf-8, and the result was different from original string but still unreadable "?e?L?X?gshiftJis-007".

Our questions are: When using C# HttpTrigger Azure function to handle Inbound Parse webhook request (request data is passed through AspNetCore.)

1. What encoding is in html/text string in Inbound Parse webhook request when the email is send in encoding other than UTF-8?
2. How to read text and html in shift_jis encoding (or other encodings excluding UTF-8) correctlyfrom an Inbound Parse webhook request?

Answer 1

Twilio Developer Evangelist here. I would recommend reaching out to the support team because it requires to investigate the payload to figure out what is going on.

I also tried to replicate the issue on my end with using send_raw option. Here's the payload, and it does contain shift_jis characters. You may be able to process the payload manually.

(stripped X-Mailer info)

'Content-Type: text/plain; charset="shift_jis"\n' +
    'X-Mailer: \n' +
    'Content-Transfer-Encoding: quoted-printable\n' +
    '\n' +
    '\n' +
    '=83e=83L=83X=83gshiftJis-007\n'