I am trying to save Dataframe as CSV file, I want to retain the spaces. I am using Spark 2.1.1, But when I try to save it as CSV file, all the spaces are trimmed.
I tried these options, but they didn't work.
option("ignoreLeadingWhiteSpace",false")
.option("ignoreTrailingWhiteSpace", "false")
Expected CSV format
SiteNumber, batch ,DayBatchDate,RecordType
190000, TBD, 12/12/2017, +00000001
My current output:
SiteNumber, batch ,DayBatchDate,RecordType
190000,TBD,12/12/2017,+00000001
ignoreLeadingWhiteSpace
andignoreTrailingWhiteSpace
options for the writer have been introduced in Spark 2.2 ( SPARK-18579) so won't have effect in Spark 2.1.