Ruby API Call Page Issue

Question

This might be a basic issue but very common so it might be helpful to other people in the future. I use HTTParty to make a get request to an API, which returns some information, something like this:

{
    datapath: "blah-blah.blah.blah",
    success: true,
    info: {
        row_limit: 500, 
        total_results: 2700,
        total_pages: 6, 
        current_page: 1
    }, 
    result: [
        {name: something, age: 1}, 
        {name: something, age: 2}, 
        {name: something, age: 3}, 
        {name: something, age: 4}, 
        {name: something, age: 5}, 
        ......
    ]
}

I want a list of all names, so I have something like this:

response = HTTParty.get('blah-blah.blah.blah')
names = response.map { |entry| entry["name"] }

With this method I only get names on the very first page aka those first 500, but I want all the names in the 2700 entries. How do I do that?

Answer 1

In fact, a quick google led me to this page https://app.enigma.io/api#data which tells me the parameter in question is limit, and that the max is 500. So, it looks like you can't get the data for all rows in one hit.

If you really NEED the data for all the rows at once (and maybe you don't, actually) then you can pass limit=500 and keep increasing the page by 1, and combine the results at your end to get all the rows.