Unable to convert UTF-16 encoded .CSV to UTF-8 in Shiny (R)

528 views Asked by At

I have been having trouble converting a UTF-16LE encoded .CSV file into UTF-8. I know that I can manually re-save the file into the desired encoding, but I want this functionality to be built into the Shiny app as my user base is not that tech savvy to even correctly save into the right file formats. This is the file they have to work with and so it is the file I have to work with too.

Ultimately, I would like to save the csv file as a data.frame() so that I can have the server wrangle the data a little bit (e.g., arranging, reshaping, etc.) before outputting a download file. I have tried the iconv() function, as well as, converting it into a JSON file with jsonlite and then converting the json into a UTF-8 encoded file.

For reproducibility

Here is a download link to the csv file I am testing with: https://www.mediafire.com/file/w4iahngpa8skgmp/InputFile.csv/file

Here is my Shiny script:

library(DT)
library(rio)
library(shiny)
library(shinydashboard)

ui <- dashboardPage(
  skin = "blue",
  dashboardHeader(title = "Title"),
  dashboardSidebar(
    sidebarMenu(
      menuItem("Import", tabName = "Import",  icon = icon("file-upload")),
      menuItem("Convert", tabName = "Convert", icon = icon("recycle"))
    )
  ),
  
  dashboardBody(
    tags$head(tags$style(HTML(
      '.myClass { 
        font-size: 20px;
        line-height: 50px;
        text-align: left;
        font-family: "Bookman Old Style",Bookman Old Style,Arial,sans-serif;
        padding: 0 15px;
        overflow: hidden;
        color: white;
      }
    '))),
    tags$script(HTML('
      $(document).ready(function() {
        $("header").find("nav").append(\'<span class="myClass"> Here is my dashboard </span>\');
      })
     ')),
    tabItems(
      tabItem(tabName = "Import",
              h2("Import CSV File", align = "center"),
              fileInput("file", "Choose CSV File",multiple = FALSE, accept = NULL,
                        width = NULL, buttonLabel = "Browse...",
                        placeholder = "No file selected"),
              actionButton("preview", "Preview Current Data Set"),
              fluidRow(
                mainPanel(
                  dataTableOutput("impOut")
                )
              )
      ),
      tabItem(tabName = "Convert",
              h2("Convert CSV File", align = "center"),
              actionButton("convert", "Convert!"),
              actionButton("preview2", "Preview Converted Data Set"),
              fluidRow(
                mainPanel(
                  dataTableOutput("conOut")
                )
              )
              
      )
    )
  ))

server <- function(input,output,server) {
  
  observeEvent(
    input$preview, 
    {
      data <- read.csv(input$file$datapath, fileEncoding = "UTF-16", sep = "\t", header = TRUE)
      output$impOut <- renderDataTable({
        datatable(data)
      })
    })
  observeEvent(
    input$convert, 
    {
      rawDat <- read.csv(input$file$datapath, fileEncoding = "UTF-16", sep = "\t", header = TRUE)
      fileDat <- iconv(rawDat, from = "UTF-16", to = "UTF-8")
      output$conOut <- renderDataTable({
        datatable(fileDat)
      })
    })
}  
shinyApp(ui,server)

Session Info:

> sessionInfo()
R version 4.1.3 (2022-03-10)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19042)

Matrix products: default

locale:
[1] LC_COLLATE=English_United States.1252  LC_CTYPE=English_United States.1252    LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C                           LC_TIME=English_United States.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] shinydashboard_0.7.2 rio_0.5.29           DT_0.22              shiny_1.7.1         

loaded via a namespace (and not attached):
 [1] zip_2.2.0         Rcpp_1.0.8.3      jquerylib_0.1.4   bslib_0.3.1       cellranger_1.1.0  compiler_4.1.3    pillar_1.7.0     
 [8] later_1.3.0       forcats_0.5.1     tools_4.1.3       digest_0.6.29     jsonlite_1.8.0    lifecycle_1.0.1   tibble_3.1.6     
[15] pkgconfig_2.0.3   rlang_1.0.2       openxlsx_4.2.5    cli_3.2.0         crosstalk_1.2.0   yaml_2.3.5        curl_4.3.2       
[22] haven_2.4.3       fastmap_1.1.0     withr_2.5.0       sass_0.4.1        vctrs_0.4.0       htmlwidgets_1.5.4 hms_1.1.1        
[29] fontawesome_0.2.2 glue_1.6.2        data.table_1.14.2 R6_2.5.1          fansi_1.0.3       readxl_1.4.0      foreign_0.8-82   
[36] magrittr_2.0.2    promises_1.2.0.1  ellipsis_0.3.2    htmltools_0.5.2   mime_0.12         xtable_1.8-4      httpuv_1.6.5     
[43] utf8_1.2.2        stringi_1.7.6     cachem_1.0.6      crayon_1.5.1   
0

There are 0 answers