How to use SqlRetryLogicBaseProvider to retry connections during Azure SQL scaling up/down

228 views Asked by At

I have an Azure SQL database, that has automatic schedules to scale up/down depending on dtu usage. During the scale up/down all connections are dropped. So I need to apply a retry logic in my application, so the application will try to execute again when the database finishing scaling up/down.

I am using C# .NET framework 4.7, with Microsoft.Data.SqlClient library that has an SqlRetryLogicBaseProvider to configure.

After configuring the provider, and calling ExecuteReader, I scale up the database, and the connection is lost, but the retry logic is never triggered, and the query just returns this error:

Cannot continue the execution because the session is in the kill state.
A severe error occurred on the current command.
The results, if any, should be discarded.

This error number is added to my error list.

Here is my code:

internal class Program
{
        private const string CnnStringFormat = "Data Source = localhost; Initial Catalog = dbtest; User ID = sa; Password=aaaaaa;Encrypt=True;TrustServerCertificate=True";
        private static SqlConnection s_generalConnection = new SqlConnection(CnnStringFormat);
        static void Main(string[] args)
        {
            SqlCommand comando = new SqlCommand("spGetLoadTest") { CommandType = System.Data.CommandType.StoredProcedure };
            
            var options = new SqlRetryLogicOption()
            {
                NumberOfTries = 5,
                MaxTimeInterval = TimeSpan.FromSeconds(60),
                DeltaTime = TimeSpan.FromSeconds(15),
                TransientErrors = new List<int> { -1, -2, 0, 109, 233, 997, 1222, 10060, -2146232060, 596 }
        };

            var provider = SqlConfigurableRetryFactory.CreateExponentialRetryProvider(options);
            
            provider.Retrying += (object s, SqlRetryingEventArgs e) =>
            {
                NotificaRetry(e);
                var cmd = s as SqlCommand;
                cmd.Connection.Close();
                cmd.Connection.Open();
                cmd.CommandTimeout = 3000;
            };

            s_generalConnection.RetryLogicProvider = provider;
            s_generalConnection.Open();

            comando.Connection = s_generalConnection;
            comando.RetryLogicProvider = provider;

            SqlDataReader leitor = null;
            try
            {
                comando.CommandTimeout = 300;
                leitor = comando.ExecuteReader();
                var count = leitor.Depth;
                var colunas = leitor.FieldCount;
            }
            catch (Exception ex)
            {
                Console.WriteLine("ERROR: " + ex.Message);
            }
        }

        private static void NotificaRetry(SqlRetryingEventArgs e)
        {
            int attempts = e.RetryCount + 1;
            Console.ForegroundColor = ConsoleColor.Yellow;
            Console.WriteLine($"attempt {attempts} - current delay time:{e.Delay} \n");
            Console.WriteLine($"error: {e.Exceptions[e.RetryCount-1].Message} \n");
            Console.ForegroundColor = ConsoleColor.DarkGray;
        }
}

I expected that during the scale up, as the line comando.ExecuteReader() is executing, the application would try again, and not return an error.

Edited

What I realised after a lot of testing is that when I get an error number 0, retry logic is never triggered, even if it is on the error list. Sometimes I got an error 109, and with error 109, retry logic is triggered. This is getting a lot of confusing to me, are there some types of error that will never trigger even if they are listed to retry?

0

There are 0 answers