I am trying to setup an SSIS package that calls a certain API. I have watched several YouTube videos, read documentation on this process. In short, the API does not provide any columns such has pages_left or hasMoreData. The max count I can pull is 100. Problem is there is WAY more than 100 records per authorization token that I have.
What is a good way to loop and hit all of the data I need? I have 20ish different authorization tokens. My last resort would be to create loops with ?page=0, ?page=1, upwards to 75 pages per token.
Example JSON data:
[
{
"Id": "00000000-0000-0000-0000-000000000000",
"Active": true,
"FirstName": "string",
"LastName": "string",
"JobTitle": "string",
"EmployerId": "00000000-0000-0000-0000-000000000000",
"StreetAddress": "string",
"City": "string",
"PostalCode": "string",
"MobileNumber": "string",
"PhoneNumber": "string",
"DateHired": "2023-07-25T17:31:54.832Z",
"EmployeeNumber": "string",
"EmergencyContact1": "string",
"EmergencyContact2": "string",
"EmergencyNotes": "string",
"CreatedOn": "2023-07-25T17:31:54.832Z",
"LastModifiedOn": "2023-07-25T17:31:54.832Z",
"Email": "string"
}
]
Attached is a screenshot showing my JSON Source Editor. I didn't know if there would be anything to put into the Request Again Rule that doesn't result in endless looping.

I am at a loss as to what I can do without creating separate loops that explicitly hit the different pages.