I'm seeking your advice to piece together a mechanism that would facilitate authentication to Dialogflow ES and CX to allow running experiments on multiple agents (projects) from our workbench application in a smooth and error-proof manner. The workbench is an internal tool written in TypeScript (using the dialogflow RPC node module) running outside of GCP. Our users analyze the results of sending the same inputs (utterances) to multiple agents, usually going back and forth between them in the course of their work.
With proper IAM configuration, we have been able to detect intents successfully by doing a gcloud auth application-default login, however we haven't found a way to update the quota project programmatically or to specify the quota project through the google.cloud.dialogflow library, so we haven't been able to fix the "switch easily between projects" part. It looks like tampering with the quota_project_id property in application_default_credentials.json once authenticated is the way to go (gcloud auth application-default set-quota-project <project>) but we would have preferred doing this programmatically.
Using service account keys (JSON) works as expected and that's what we have been doing so far, that's also what we do in our CI/CD pipeline and in our agents running in production. But we aim at reducing the amount of service account credentials file that we share with individuals. Ideally, speech/data scientists would use their own end-user credentials to perform experiments.
We are looking for alternatives so that users would authenticate once with gcloud auth application-default login and the workbench would handle the rest behind the scenes, using only, as additional argument, the project-id against which the experiment must be run. This would eliminate the need to pause the experiment to update the quota project (using set-quota-project), or to update the GOOGLE_APPLICATION_CREDENTIALS variable when using service account keys.
Another thing we tried was Service Account Impersonation, unfortunately this does not seem supported by the google.cloud.dialogflow library, so even though we were able to successfully submit requests (with Curl/Postman) to the Dialogflow RESTful API using impersonation, we haven’t been able to leverage this mechanism in our code.
Has anyone been able to overcome a similar challenge? Is there any other authentication mechanism that could help us achieve this goal?