How to loop through pagination for a shopify api request in ASP.NET Core 7 with HttpClient

Question

I am trying to get all the orders from a shopify rest API but they limit their response to a certain number of orders via pagination. I am trying to loop through the pagination asynchronously without hitting their limit (there is a header in the response called: X-Shopify-Shop-Api-Call-Limit: 1/40).

What is the best way to approach this?

So far I am able to do this via a do while loop with their since_id but synchronously since it requires the last id of the last response:

    [HttpGet(Name = "GetShopifyOrderData")]
[EnableRateLimiting("api")]
public async Task<IActionResult> Get()
{
    var shopifyOrders = new List<ShopifyOrder>();
    long LastId = 0;
    bool HasMorePages = true;
    try
    {
        do
        {
            var httpRequestMessage = new HttpRequestMessage(
                HttpMethod.Get,
            $"https://mysite.myshopify.com/admin/api/2023-07/orders.json?limit=100&since_id={LastId}")
            {
                Headers =
        {
            { "X-Shopify-Access-Token", _shopifyAPIAuth.AccessToken }
        }
            };

            var httpClient = _httpClientFactory.CreateClient();
            var httpResponseMessage = await httpClient.SendAsync(httpRequestMessage);

            if (httpResponseMessage.IsSuccessStatusCode)
            {
                var content = await httpResponseMessage.Content.ReadAsStringAsync();
                var shopifyResponse = JsonSerializer.Deserialize<ShopifyAPI.ShopifyOrderResponse>(content);
                if (shopifyResponse?.Orders != null)
                {
                    shopifyOrders.AddRange(shopifyResponse.Orders);
                    if (shopifyResponse.Orders.Count == 0)
                    {
                        HasMorePages = false;
                    }
                    else
                    {
                        LastId = shopifyResponse.Orders[shopifyResponse.Orders.Count - 1].Id;
                    }
                }

            }
        } while (HasMorePages);
        return Ok(shopifyOrders);

    }
    catch (Exception ex)
    {
        _logger.LogInformation("Message: Error: " + ex.ToString(), DateTime.UtcNow.ToLongTimeString());
        throw;
    }
}

Their response is wrapped in an orders property in the JSON so really ShopifyOrderResponse contains the following model:

    public class ShopifyOrderResponse
    {
        [JsonPropertyName("orders")]
        public List<ShopifyOrder>? Orders { get; set; }
    }

I know I could create a list variable and keep adding the range of the responses that come back but I am not sure how I should do the loop with a HttpClient to ensure it's thread-safe but still asynchronous.

I also do not know how I can use the next link in the response headers:

link: <https://mysite.myshopify.com/admin/api/2023-10/orders.json?limit=100&page_info=fdsafasdfasdfasdfasdf>; rel="next"

to run the next request.