Create new table with bigrquery

63 views Asked by At

I am trying to create a new table in Bigquery, using bigrquery and tables already loaded in bigquery.

I know I could do:

library(tidyverse)
library(bigrquery)
library(DBI)

project_id <- "my_project"
dataset_id <- "my_dataset"

con <- dbConnect(
  bigrquery::bigquery(),
  project = project_id,
  billing = project_id,
  dataset = dataset_id
)

my_df <- tbl(con, "mytable") %>%
   some_manipulation %>%
   collect()     

bq_table_upload("project.dataset.table_output", my_df)

But, I want to avoid downloading and uploading the data.

I could just send a query with DBI::dbGetQuery(con, "CREATE TABLE AS..."), but I insist, my intention is to manipulate tables within bigquery with bigrquery making use of tidyverse verbs.

Finally, I have also tried:

library(tidyverse)
library(bigrquery)
library(DBI)

project_id <- "my_project"
dataset_id <- "my_dataset"
table_output <- "table_output"

bq_perform_query(
  my_query,
  project = project_id,
  destination_table = bq_table(project = project_id, dataset = dataset_id, table = table_output),
  billing = project_id 
)

I donĀ“t get any error when executing, but nothing happens on bigquery side, I cannot see my table_output in the dataset.

Is there a way to send the query to bigquery and create a new table without downloading, collecting and uploading data? It seems to me a pretty common operation.

0

There are 0 answers