Unable to set up Export Collections to BigQuery extension. Unable to uninstall: failed to remove IAM roles from Service Account

Question

I tried installing the Export Collections to BigQuery extension to send firestore collection data to BigQuery. The installation succeeded but could never get the queries to work. I kept getting the error:

Not found: Dataset myproject:firestore_events was not found in location US

I tried uninstalling and reinstalling. I now have two instances of the extension and attempts to uninstall fail with ther error:

failed to remove IAM roles from Service Account for projects/myproject/instances/firestore-bigquery-export

Is there a way to uninstall manually? Is there a way to install manually?

Answer 1

The following error message means that your FROM statement may not contain the proper query structure - project, database, and table:

Not found: Dataset myproject:firestore_events was not found in location US

For example:

SELECT *
FROM `my-project.my-database.my-table`

Please note the backticks.

Refer to Error: Not found: Dataset my-project-name:domain_public was not found in location US for details about a similar issue.

Now, in regards to the second error message:

failed to remove IAM roles from Service Account for projects/myproject/instances/firestore-bigquery-export

I suspect that after reinstalling and uninstalling one of the extension instances, its service account might already be deleted and thus failing to remove its IAM roles.

The installation of this extension generates a new service account that has the BigQuery Data Editor role to access your project and resources. This role allows the extension to configure and export data into BigQuery.

From documentation:

When you uninstall an extension from your project, Firebase deletes the service account created for that instance of the extension. After this deletion of the service account, the extension cannot run in your project because it no longer has any access rights to your project or data.

I was able to uninstall and reinstall the Export Collections to BigQuery extension to avoid duplicates in my project without any issues.

To answer your last question, I am not aware of any other methods to install/uninstall Firebase extensions other than what is shown in the documentation. You can install the extension using the Firebase Console or the Firebase CLI:

1 - Install the Firebase CLI or update it to the latest version:

npm install -g firebase-tools

2 - Install the extension:

firebase ext:install firestore-bigquery-export --project=projectId_or_alias

Edited

We can take the following steps to troubleshoot error:

failed to remove IAM roles from Service Account for projects

• Please make sure that you're the owner of the project that you are installing extensions in.
• Refer to this GitHub issue to uninstall the extension using the CLI.
• If that didn't help, try disabling and re-enabling the Firebase Extensions API inside the GCP Console.
• Please make sure that Firebase Management API is enabled in your project.
• If that didn't help, navigate to the IAM Permissions page on the Cloud Console, and find the service account.
  • Add the Cloud Functions Developer and Firebase Extensions API Service Agent roles.
    • If those roles not available for some reason, just adding the Project Editor role also works as a catch-all. However, using the catch-all also provides additional permissions to the service accounts.

Answer 2

I had a similar issue when installing a Firebase Extension.

My error was adding an IAM role that did not exist, this not only failed to install the extension, but also then stopped me from uninstalling with the error failed to remove IAM roles from Service Account for.

To fix this issue I edited what I knew was the broken permission to be the correct role and then ran the following...

firebase ext:update ${ext-name} . --project={project-name}.

Alternativley, removing all roles completetly from the configuration and updating will have the same result.

One completed, the uninstallation process will automatically continue and uninstall your extension.