Terraform claims backend GCS bucket doesn't exist

Question

Terraform 1.4.5 on Windows, authenticating as individual user via gcloud auth application-default login. I suddenly started getting this error a few hours whenever I do any operating involving the state file stored in a Google Cloud Storage bucket:

Error: Failed to get existing workspaces: querying Cloud Storage failed: storage: bucket doesn't exist

My backend.tf file has not changed in several months. It simply lists the GCS bucket name and prefix:

terraform {
  backend "gcs" {
    bucket = "my-bucket-name"
    prefix = "my-module-name"
  }
}

I've tried the usual fixes, such as running terraform init -reconfigure and deleting the .terraform hidden directory, but always get the same error message.

This did happen around the time there were some permission changes and cleanup tasks done by the group that manages our GCP projects. But, the bucket still exists and I have read/write access to it.

Answer 1

Found the root cause of this error is gcloud's default project had been deleted. This doesn't affect Terraform directly, but was influencing the quota project, which can be viewed in file $HOME/.config/gcloud/application_default_credentials.json (Linux/Mac) or $env:APPDATA\gcloud\application_default_credentials.json (Windows)

To fix, I did this:

gcloud auth login
gcloud config set project my-main-project-id
gcloud auth application-default login

This should also do the trick:

gcloud auth application-default set-quota-project my-main-project-id