Automatic job to delete bigquery table records

Question

Is there a way to schedule deletion of rows from bigquery table based on a column condition? Something like a job to schedule to run every day.

For example, let's say I've a column called creation_date in the table. I need to delete records when creation_date is less than current date minus one week (creation_date < current date - 7). I need the job to run everyday on a specified time and delete records based on the creation date condition.

If there aren't any built in scheduler operations, could you suggest any options available?

Answer 1

You have a couple simple options within BigQuery itself you can utilize.

The simplest is likely scheduled queries. This will simply just execute a command on a schedule. You can execute a DELETE statement or some other method.

Additionally you could set table or partition expirations. This one involves a little more legwork but would achieve a similar result. Based on your description it would likely be a partition expiration you would want to set up.