DuckDB: querying csvs containing values with newlines

129 views Asked by At

According to the csv spec, values containing newlines should be wrapped in double quotes. Spreadsheet programs generally handle this well, treating a newline within a quoted value as a single cell.

Unfortunately I haven't managed to achieve the same in DuckDb. If I try to select using read_csv_auto, upon scanning the value with a newline DuckDb will raise an error on the grounds of an unterminated double quote.

I can escape the newline by hand, e.g. \\n, but this will still resolve to \\n in query output, so I'd also need to manually unescape string-typed fields which quickly becomes a pain to maintain. I can change the newline character, but since the only options are \n, \r, and \r\n, this only really moves the problem.

Is there a good solution here for getting back the same newlines I insert, without raising errors?

0

There are 0 answers