Permission bigquery.tables.get denied or it may not exist

Question

I am using the AWS Glue connector for BigQuery. My glue jobs were running fine in multiple AWS accounts but suddenly it started failing with the below response in all the accounts together:

Access Denied: Table common-infra-services:detailedcost.gcp_billing_export_resource_v1_01E8AD_3E792E_BB0E5D: Permission bigquery.tables.get denied on table common-infra-services:detailedcost.gcp_billing_export_resource_v1_01E8AD_3E792E_BB0E5D (or it may not exist).", "reason": "accessDenied"

Please review and let me know what could be the issue of this problem.

I am using the GCP IAM service account role to run queries using Glue to BigQuery with the following set of permissions:

bigquery.jobs.create
bigquery.tables.getData
bigquery.tables.list    

And with these permissions, all jobs were running fine till yesterday.

Answer 1

Based on that error message I'd check if table common-infra-services:detailedcost.gcp_billing_export_resource_v1_01E8AD_3E792E_BB0E5D exists. If it does you might need to add permission bigquery.tables.get to your service account.