I'm getting the Freetext Answer truncated when there are special characters. Before submission, we make sure to validate the answer which is in JSON form by stringifying and parsing. But when we retrieve the answer from MTurk, it gets truncated around emojis, quotation marks, arabic characters etc, so we don't get the full response. I'm not able to replicate in the Workers Sandbox when sending in answers with those special characters though.
I noticed that the answer's XML encoding is in ASCII despite referring to the Mturk Schema for QuestionFormAnswers which is UTF-8 encoded. The Question does not specify an encoding so I'm assuming it's correctly UTF-8.
"Answer": "<?xml version=\"1.0\" encoding=\"ASCII\"?><QuestionFormAnswers xmlns=\"http://mechanicalturk.amazonaws.com/AWSMechanicalTurkDataSchemas/2005-10-01/QuestionFormAnswers.xsd\"><Answer>...
"Question": "<?xml version=\"1.0\"?>\n<HTMLQuestion xmlns=\"http://mechanicalturk.amazonaws.com/AWSMechanicalTurkDataSchemas/2011-11-11/HTMLQuestion.xsd\">\n <HTMLContent>...
I could not get access to our AWS account, and I can't create a Requester's account in our country to check the UI.
I wanted to test whether setting the Answer's encoding to UTF-8 will resolve the issue, before trying it out encoding/decoding given my limited access.