Encoding conversion of a fetch response

Question

Inside a React Native method I'm fetching a xml encoded in ISO-8859-1.

As long as the fetching is completed I'm trying to convert it to UTF-8.

Here the code:

const iconv = require('iconv-lite');

fetch('http://www.band.uol.com.br/rss/colunista_64.xml', {
      headers: {
        "Content-type": "text/xml; charset=ISO-8859-1"
      }
})
.then(res=>res.text()})
.then(text => {
   const decodedText = iconv.decode(Buffer.from(text, 'latin1'), 'latin1')
  , output = iconv.encode(decodedText, 'utf8')
   console.log(output.toString())
})

The problem is: all especial characters of the body is being replaced by "¿½"

For the conversion, I'm using the package iconv-lite

What is the better workaround for this problem?

Answer 1

The best workaround is to use res.arrayBuffer() instead res.text(), as long the Buffer constructor accepts ArrayBuffer

The code:

fetch('http://www.band.uol.com.br/rss/colunista_64.xml')
      .then(res => res.arrayBuffer())
      .then(arrayBuffer => iconv.decode(new Buffer(arrayBuffer), 'iso-8859-1').toString())
      .then(converted => console.log(converted))

Answer 2

let axiosConfig = {
    responseType: 'arraybuffer',
    responseEncoding: 'binary',
  }

const htmlPage = await axios.get(`http://www.band.uol.com.br/rss/colunista_64.xml`, axiosConfig);
const decoder = new TextDecoder('iso-8859-1');
decoder.decode(pageHtml.data)

Answer 3

The solution is actually rather simple. Just use axios instead of fetch. Axios works out of the box in react native, without any special dependencies. It converts the encoding of a response to UTF-8 automatically for each request by default without any configuration.

npm install axios

Example:

import axios from 'axios';

let response = await axios.get(url);
console.log(response);

Thats it! The response.data is auto converted to UTF-8 from whichever encoding the archaic site uses.

Answer 4

As pointed out by Hellon Canella Machado you can't use res.text() and must use an ArrayBuffer as a workaround.

Since res.arrayBuffer() doesn't work with fetch in React Native you can use the XMLHttpRequest API.

import iconv from 'iconv-lite';
import { Buffer } from 'buffer';

function fetchXML() {
  return new Promise((resolve, reject) => {
    const request = new XMLHttpRequest();

    request.onload = () => {
      if (request.status === 200) {
        resolve(iconv.decode(Buffer.from(request.response), 'iso-8859-1'));
      } else {
        reject(new Error(request.statusText));
      }
    };
    request.onerror = () => reject(new Error(request.statusText));
    request.responseType = 'arraybuffer';

    request.open('GET', 'http://www.band.uol.com.br/rss/colunista_64.xml');
    request.setRequestHeader('Content-type', 'text/xml; charset=ISO-8859-1');
    request.send();
  });
}

fetchXML().then(response => 
    console.log(response)
);

Also make sure that you have the packages buffer and stream installed.