How to do paged queries in RETS?

Question

I am using a java client (jrets) to query a RETS server. This rets server does not support the OFFSET functionality.

The server admin told me to use the MODIFIED field as a way to page through the records. However, I found that this process does not work.

My use case:

The use case is to download all records of a certain type since january of this year. So, I issue a query with a limit of 500 and download records. I go through the data, and get the maximum MODIFIED date. Then I issue another query from that point on.

However I find that the server is not sorting the data by MODIFIED date and returning it. So, data in each batch contains gaps.

Is there a better way to achieve what I want to do?

Answer 1

RETS doesn't support sorting. The server admin probably meant for you to query using date ranges.

The RETS server should support date range queries like (date_field=2009-01-01-2009-01-25) or even with the time such as (MODIFIED=2013-08-01T00:00:00-2013-08-31T23:59:59).

In this manner you can work out a batch of queries you need to send to retrieve all the listings. Your timezone you set in the script will have an affect so you'll need to know what the server time is. Some will use UTC/GMC and some will use their local time.

Here's more help, https://www.flexmls.com/developers/rets/tutorials/dmql-tutorial/

Answer 2

I am not sure if this is universal but the GAMLS RETS sends responses sorted by L_ListingID.

I can start with

query = "(L_UpdateDate=2017-01-01T11:42:59Z+)"

Then find the largest 'L_ListingID in the response.

Build a new query

query = "(L_UpdateDate=2017-01-01T11:42:59Z+),(L_ListingID=[LARGEST_Listing_ID])"

And keep going until I get an empty response.