xml2::read_html with proper character encoding crashes on Ubuntu

Question

xml2::read_html crushes on Ubuntu (but not on a Mac) when trying to use correct character encoding.

   library(xml2)
   library(httr) 
   # GET webpage that is encoded using Big5 (Chinese)
   pg <- GET("http://chinesenews.net.au")
   # Identify encoding using rvest package function, which returns 
   # incorrect encoding as ISO-8859-1
   enc1 <- rvest::guess_encoding(httr::content(pg, "raw"))$encoding[1]
   # Use hack to identify the right encoding using a function from stringi package
   enc2 <- as.character(
                as.data.frame(
                      stringi::stri_enc_detect(httr::content(pg, "raw"))[[1]])[1,1]) 
   # So far so good. 
   # Let's try to read_html with both encodings
   # Using ISO-8859-1 encoding, there is not problem
   ht1 <- xml2::read_html(pg, encoding=enc1) # Reads, but characters are distorted
   # However, using correct (Big5) encoding crashes on Ubuntu
   ht2 <- xml2::read_html(pg, encoding=enc2) 

The error is:

Error in doc_parse_raw(x, encoding = encoding, base_url = base_url, as_html = as_html, : basic_string::_M_replace_aux

Because the problem happens on Ubuntu but not on Mac, tried to install the latest version of xml2 library using

devtools::install_github("hadley/xml2")

There is still an error, although a different one:

Error in doc_parse_raw(x, encoding = encoding, base_url = base_url, as_html = as_html, : input conversion failed due to input error, bytes 0xFB 0x7C 0xB7 0x51 [6003]

I am not sure why passing the right encoding crashes the libxlm2. Any ideas what can be done?

Here is my Ubuntu sessionInfo():

R version 3.2.3 (2015-12-10)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 16.04.1 LTS

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods  
[7] base     

other attached packages:
[1] httr_1.2.1   xml2_1.0.0   magrittr_1.5

loaded via a namespace (and not attached):
 [1] selectr_0.3-1   R6_2.2.0        tools_3.2.3     curl_2.3       
 [5] urltools_1.6.0  Rcpp_0.12.8     triebeard_0.3.0 stringi_1.1.2  
 [9] stringr_1.1.0   rvest_0.3.2     purrr_0.2.2   

Answer 1

Try with encoding = "latin1".