HTTPS POST in shift-jis encoding

Question

I'm trying to send a POST request to a server which decodes with SHIFT-JIS. This string サービス is being translated to 繧ｵ繝ｼ繝薙せ after being decoded in SHIFT-JIS. It seems like the request will always be encoded in UTF-8 whenever the request is being sent over. I'm using nodejs for posting the request. Question is how do I send over the characters in shift-jis encoding? It seemed easy but I just couldn't find out how to.

Listening server

var iconv = require('iconv-lite');
const http = require('http');

http.createServer((request, response) =>
{
    const
    {
        headers,
        method,
        url
    } = request;
    let body = [];
    request.on('error', (err) =>
    {
        console.error(err);
    }
    ).on('data', (chunk) =>
    {
        body.push(chunk);
    }
    ).on('end', () =>
    {
        body = Buffer.concat(body).toString();
        // BEGINNING OF NEW STUFF
        body = iconv.decode(Buffer.from(body), 'shift_jis');
        response.on('error', (err) =>
        {
            console.error(err);
        }
        );

        response.statusCode = 200;
        response.setHeader('Content-Type', 'application/json');
        // Note: the 2 lines above could be replaced with this next one:
        // response.writeHead(200, {'Content-Type': 'application/json'})

        const responseBody =
        {
            headers,
            method,
            url,
            body
        };

        response.write(JSON.stringify(responseBody));
        console.log(body);
        console.log(responseBody);
        response.end();
        // Note: the 2 lines above could be replaced with this next one:
        // response.end(JSON.stringify(responseBody))

        // END OF NEW STUFF
    }
    );
}
).listen(8000);

Request

var request = require('request');

request({
    url: 'http://localhost:8000',
headers: { 'Content-Type': 'application/x-www-form-urlencoded; charset=shift_jis' },
    method: 'POST',
    body: 'サービス'
}, function(error, response, body){
    if(error) {
        console.log(error);
    } else {
        console.log(response.statusCode, body);
    }
});

EDIT: It turns out that the axios module we're using for HTTPS POST will encode the payload in UTF-8 before sending out the request. We cloned the axios module and modifying it to encode in SHIFT-JIS instead.

Answer 1

If you want to send a string with Shift-JIS encoding, you have to convert the target string (it is represented in UTF-16 internally) into Shift-JIS before adding it into the request body.

The standard TextEncoder only supports UTF-8 encoding and cannot handle Shift-JIS encoding. So you have to use additional modules like encoding.js or text-encoding for this purpose.