Scala: Replacing double quotes with single quotes

Question

How do you replace single quotes with double quotes in Scala? I have a data file that has some records with "abc" (double quotes). I need to replace these quotes with single quotes and convert it to a data frame.

val customSchema_1 =        
  StructType(Array(
  StructField("ID", StringType, true),
  StructField("KEY", StringType, true),
  StructField("CODE", StringType, true))

val df_1 = sqlContext.read
  .format("com.databricks.spark.csv")
  .option("delimiter", "¦")
  .schema(customSchema_1)
  .load("example")

Answer 1

Read line by line your file and apply following example to each of them:

val text: String = """Here is a lot of text and "quotes" so you may think that everything is ok until you see something "special" or "weird"
"""

text.replaceAll("\"", "'")

This will give you a new String value with quotes instead of double quotes.

Answer 2

You can create a simple udf to replace double quote with single quote

Here is a simple example

import org.apache.spark.sql.functions.udf

val removeDoubleQuotes = udf( (x:String) => s.replace("\"","'"))

//If df is the dataframe and use the udf to colName to replace " with '

df.withColumn("colName", removeDoubleQuotes($"colName"))

Hope this helps!