Parquet write to gcs is not queryable by bigquery in nodejs

194 views Asked by At

i'm using parquetjs to create parquet files and push to google cloud storage.

Problem is that bigquery cannot read the data from file but when i use parquet-tools everything looks healthy.

1

There are 1 answers

1
Mahan Hazrati Sagharchi On BEST ANSWER

just pass useDataPageV2: false as option to parquet.ParquetWriter.openFile(...)

Like this:

const writer = await parquet.ParquetWriter.openFile(schema, filename, { useDataPageV2: false })