Insert timestamp into bigquery table using pandas

2.5k views Asked by At

I have a table in Google bigquery, where a column is set to datatype timestamp.

I have to insert data using to_gbq function of pandas.

If I set the datatype to string instead of timestamp the data is loaded into the table.

But I want the column to be timestamp datatype.

How can the column type of dataframe converted to timestamp compatible with Google bigquery.

Error

enter image description here

Table schema

enter image description here

1

There are 1 answers

6
iDrwish On

Timestamp is stored as int64 datatype, or int, so if you would like to format it in such a way, you can store cast your column to this datatype in pandas.

import numpy as np
import pandas as pd
df.COLUMN_OBJECT = df.COLUMN_OBJECT.astype(np.int64)
df.COLUMN_DATETIME = df.COLUMN_DATETIME.apply(lambda x: x.timestamp).astype(np.int64)

You can have the timestamp in seconds, ms, or ns.