How to load multiple files (same schema) from LOCAL into a table in BigQuery?

873 views Asked by At

I have multi-files in a folder in my local machine. And every file has the same schema. how can I upload those files into BQ in one line cli? I tried this:

bq load --source_format=NEWLINE_DELIMITED_JSON --ignore_unknown_values temp.test_load ./* ./schema.json

But I got this error Too many positional args, still have...(and continues with the name of all files in the folder)

But when I specife the file name, it uploads it into BQ without any error:

bq load --source_format=NEWLINE_DELIMITED_JSON --ignore_unknown_values temp.test_load ./file_1.ndjson.gz ./schema.json (this one is working)

How can I make multi uploads from local?

1

There are 1 answers

0
Ivana S On

When loading data from local files to BigQuery, files can only be loaded individually as the Wildcards and comma-separated lists are not supported for loading data from local files.

The Wildcards are only supported if you are loading data from Cloud Storage to BigQuery, provided all the files share a common base-name.