swift euc-kr korean encoding not working. But works in python

Question

I am writing some code to parse korean text from server encoded with euc-kr korean encoder.

When I just do the same encoding in Python, it works as expected. But when I do it as following, encoding doesn't work. The result is unreadable.

In Python :

string = u'안녕하세요.'.encode('eucKR') 

In Swift :

let encoding:UInt =  CFStringConvertEncodingToNSStringEncoding(CFStringEncoding(
        CFStringEncodings.EUC_KR.rawValue))

let encodedData = "안녕하세요.".data(using: String.Encoding(rawValue: encoding))!

What the difference between those 2 encodings ?

Following are full source codes for both python and swift. I still stuck on the encoding part. Is the problem related to alamofire post request?

Python:

import requests
from pattern import web

string = u'저는 내일 바빠서 학교에 못갑니다.'.encode('eucKR')
r = requests.post("http://nlp.korea.ac.kr/~demo/dglee/komatag.php", data={'formradio1': '', 'formradio2': 'ems', 'textarea': string})
dom = web.Element(r.text)
main = dom('tr')
for item in main:
    result = web.plaintext(item.source)
    a = result.encode('ISO-8859-1')
    t=a.decode('eucKR')
    print(t)

Swift:

    override func viewDidLoad() {

        let string: NSString = NSString(string: "안녕하세요")
        let encodedEucKr = stringToEuckrString(stringValue: string as String)
        print(encodedEucKr)

        Alamofire.request("http://nlp.korea.ac.kr/~demo/dglee/komatag.php", method: .post, parameters: ["formradio1":"", "formradio2":"ems", "textarea": encodedEucKr], headers: nil).responseString { response in

            switch(response.result) {
            case .success(_):
                if let data = response.result.value{
                    print(response.result.value)
                }
                break

            case .failure(_):
                print(response.result.error)
                break

            }
        }

    }


func stringToEuckrString(stringValue: String) -> String {

    let encoding:UInt = CFStringConvertEncodingToNSStringEncoding(CFStringEncoding(
        CFStringEncodings.EUC_KR.rawValue))

    let encodedData = stringValue.data(using: String.Encoding(rawValue: encoding))!

    let attributedString = try? NSAttributedString(data: encodedData, options:[:],        documentAttributes: nil)

    if let _ = attributedString {
        return attributedString!.string
    } else {
        return ""
    }
}

Answer 1

It was not easy for two reasons...

1. Sending form data in EUC-KR is not considered to be standard-compliant in modern web technologies and standards.

2. The response sent from your server is sort of broken, in that Swift cannot decode the result as a valid EUC-KR text.

   (This seems to be a bug of your server side code.)

Anyway, when you need to send a web form based request to your server in EUC-KR:

• Create a EUC-KR byte sequence from the original
• Percent-escape it. You may need to do it by yourself
• Put entire request in an HTTP request body
• Add proper MIME type header

Some details depend on the server. I have never used Alamofire, so I do not know if Alamofire supports such things.

Here I show you an example using a normal URLSession:

override func viewDidLoad() {
    super.viewDidLoad()
    // Do any additional setup after loading the view, typically from a nib.
    sendRequest(string: "안녕하세요")
}

func sendRequest(string: String) {
    let rawEncoding = CFStringConvertEncodingToNSStringEncoding(CFStringEncoding(CFStringEncodings.EUC_KR.rawValue))
    let encoding = String.Encoding(rawValue: rawEncoding)
    
    let url = URL(string: "http://nlp.korea.ac.kr/~demo/dglee/komatag.php")!
    var request = URLRequest(url: url)
    request.httpMethod = "POST"
    //Create an EUC-KR byte sequece
    let eucKRStringData = string.data(using: encoding) ?? Data()
    //Percent-escape, you need to do it by yourself
    //(Though, most servers accept non-escaped binary data with its own rules...)
    let eucKRStringPercentEscaped = eucKRStringData.map {byte->String in
        if byte >= UInt8(ascii: "A") && byte <= UInt8(ascii: "Z")
        || byte >= UInt8(ascii: "a") && byte <= UInt8(ascii: "z")
        || byte >= UInt8(ascii: "0") && byte <= UInt8(ascii: "9")
        || byte == UInt8(ascii: "_") || byte == UInt8(ascii: ".") || byte == UInt8(ascii: "-")
        {
            return String(Character(UnicodeScalar(UInt32(byte))!))
        } else if byte == UInt8(ascii: " ") {
            return "+"
        } else {
            return String(format: "%%%02X", byte)
        }
    }.joined()
    //In application/x-www-form-urlencoded format, you send data in a URL-query like format.
    let paramString = "formradio1=&formradio2=ems&textarea=\(eucKRStringPercentEscaped)"
    //As all non-ASCII characters are percent-escaped, .isoLatin1 works well here.
    let bodyData = paramString.data(using: .isoLatin1)!
    //Form data needs to be sent as a body of HTTP protocol.
    request.httpBody = bodyData
    //MIME type for usual form data is "application/x-www-form-urlencoded".
    request.addValue("application/x-www-form-urlencoded", forHTTPHeaderField: "Content-Type")
    //URLRequest is ready and you can start dataTask here.
    let task = URLSession.shared.dataTask(with: request) {data, response, error in
        if let error = error {
            print("Error:", error)
        }
        if let response = response {
            print("Response:", response)
        }
        //The response may not be valid EUC-KR; you need to decode it while accepting invalid bytes.
        if let data = data {
            var result = ""
            var i = 0
            while i < data.count{
                let ch = data[i]
                if ch < 0x80 {
                    result += String(Character(UnicodeScalar(UInt32(ch))!))
                } else if
                    i + 2 <= data.count,
                    let ch2 = String(data: data.subdata(in: i..<i+2), encoding: encoding)
                {
                    result += ch2
                    i += 1
                } else {
                    result += "?"
                }
                i += 1
            }
            print("Result:", result)
        }
    }
    //Do not forget to resume the created task.
    task.resume()
    //And remember you should not do anything after you invoke an async task.
}

If your server side can handle UTF-8 requests and responses properly, the code above can be far more simple. Using EUC-KR in web services is sort of outdated. You'd better adopt UTF-8 soon.