GCS - Secure Way to Authenticate to GCS From On-Premises Server

Question

I am looking for the secure/recommended way to connect and perform actions on GCS.

Need:

We have files on an on-premises Linux environment that needs to be uploaded to GCS on a hourly basis. Newer files alone will need to be uploaded. Multiple users/processes use the server. Manual intervention or authentication is not possible. Ok to have one-time authentication setup.

Current Implementation:

A simple shell script using service account as shown below. Script is authenticated using the JSON key file of the service account with access to the storage account.

gsutil -m rsync -r -P -e <LINUX_PATH> <GCS_PATH>

Questions:

1. What is the safest and recommended way to authenticate from an on-premises linux server outside of GCP to connect to GCS?

2. Is there a better way to handle the authentication without downloading the key file? I understand that if an intruder gets access to the server, we are done for anyways, but still would be nice if we had a better option.

Update:

In case Service Account Key file is the only options, please suggest the recommended way to secure the file. Currently we limit the permission of the file to the user executing the shell script. Any other options would be appreciated.

3. We had considered "Service Account Impersonation". But in this case, how are we supposed to login to the user having only the serviceAccountTokenCreator role. We would once again need to resort to JSON key file?

3.1. In case of Service Account Impersonation, is the allowed time defaulted to 60 minutes?

Answer 1

There are different ways you can do uploads in GCS. As to your script, you are using single-request upload type. You may want to check the following ways:

• Resumable uploads, which can use OAuth 2.0 and XML configuration.

• Streaming uploads, which uses checksum validation when streaming.

Just remember that everytime additional security that you configure with your system, there can be added complexity on implementing it.

Answer 2

Storing the service account key in a file is the simplest solution, but you have risks associated with access to that file.

If your Linux machine uses another identity service, you should look into Workload Identity Federation. With it the authentication task moves to your identity provider, and on Google you grant that identity access to Google Cloud resources, or allow it to impersonate a service account. No service account key file required.