Does Bigquery Storage Write API Create a new connection for each append_row in default stream?

Question

When I check the Concurrent connections quota for Bigquery Storage Write API, it says 1,000 in a region; 10,000 in a multi-region. This number seems very small. Is this limit per project or per stream? And if I append_rows to the same default stream from different instances at the same time will those count towards the connection limit?

Answer 1

The Concurrent connection quota is applied per project where the stream is initiated.

The concurrent connections quota is based on the client project that initiates the Storage Write API request

The initiating project is the project associated with the API key or the service account

Basically, the default stream reuses a single connection for all the append_row requests within a stream, however as I understand your use case, these instances will fall under the Application-created streams which can only have a single connection hence initializing an append_row requests from multiple instances to the same default stream in effect creates multiple connections that shares the same connection limit.

You can review the BigQuery Storage Write API best practices to understand more about the relationship between a stream and a connection.