How do you convert an ndjson file to a csv file in Python?

Question

I currently have my file loaded as follows:

import ndjson

with open('full_simplified_arm.ndjson') as f:
    data = ndjson.load(f)

However, I would like to convert this file into a csv in Python and was wondering how to do this? Thanks

Answer 1

You could do it with csv.DictWriter.

Loading the data with ndjson.load, it is stored in this form.

[{'id': 1, 'name': 'Alice'}, {'id': 2, 'name': 'Bob'}, {'id': 3, 'name': 'Carol'}]

You could take the keys from the first item in the list as the fieldnames of the table.

fields = data[0].keys()

Then define a csv.DictWriter to write the header and the rows.

writer = csv.DictWriter(f, fieldnames=fields)
writer.writeheader()
for item in data:
    writer.writerow(item)

Complete solution:

import csv
import ndjson

with open('./users.ndjson', 'r') as f:
    data = ndjson.load(f)

with open('./users.csv', 'w', newline='') as f:
    # I'm assuming that each item has the same keys
    fields = data[0].keys()
    writer = csv.DictWriter(f, fieldnames=fields)
    writer.writeheader()
    for item in data:
        writer.writerow(item)

Input

{"id":1,"name":"Alice"}
{"id":2,"name":"Bob"}
{"id":3,"name":"Carol"}

Output

id,name
1,Alice
2,Bob
3,Carol

Note: The sample data was retrieved online