Why odata Trippinservice returns only 8 pages?

Question

When I give the odata service url,https://services.odata.org/V4/(S(ysqt4lcalbsipb1qkoc04ryb))/TripPinServiceRW/People in browser,I only get 8 records.Why do I get only 8 records when there are total of 20 records in People entity?Is PageSize set in Trippinservice?Can anyone help me to understand this?

Answer 1

Yes, this service implements server-side paging.

Firstly we identify that server-side pagination is in effect from the presense of the @odata.nextLink property in the response, this is in the root of the response:

{
    "@odata.context": "http://services.odata.org/V4/(S(ysqt4lcalbsipb1qkoc04ryb))/TripPinServiceRW/$metadata#People",
    "@odata.nextLink": "https://services.odata.org/V4/(S(ysqt4lcalbsipb1qkoc04ryb))/TripPinServiceRW/People?%24skiptoken=8",
    "value": [
        ... 8 records ... 
    ]
}

We can only assUme that the page size is 8 by counting the number of records in the response or by consulting the documentation for the given service. In this case there is a $skiptoken query parameter inside the next link, in this case it happens to have a value of 8 and this corresponds to the number of records, but only by coincidence.
NOTE: %24 is a dollar sign $ that has been url encoded

~/TripPinServiceRW/People?%24skiptoken=8

In the case of the TripPin service documented in the OData Basic Tutorial, the $skiptoken value represents the number of records to skip and is a common implementation, but it is not a standard.

There is no specific mention of the server page size for the People feed in the documentation, nor is there a standard way to document $skiptoken logic at all in the $metatdata. Have a read on how and why ASP.NET WebAPI implements skiptoken: Use $skiptoken for server-driven paging

We can demonstrate this by navigating the next nextLink or altering the $orderby:

GET: ~/TripPinServiceRW/People?$skiptoken=8

{
    "@odata.context": "http://services.odata.org/V4/(S(ysqt4lcalbsipb1qkoc04ryb))/TripPinServiceRW/$metadata#People",
    "@odata.nextLink": "https://services.odata.org/V4/(S(ysqt4lcalbsipb1qkoc04ryb))/TripPinServiceRW/People?%24skiptoken=16",
    "value": [
        ... 8 more records ...
    ]
}

The $skiptoken is now 16 in the new nextLink: ~/TripPinServiceRW/People?%24skiptoken=16. In many implementations the $skiptoken will represent the key value of the last record that was sent, but TripPin does not use this convention, we can verify that by changing the order:

Both of these responses will have $skiptoken=8 in the nextLink

GET: ~/TripPinServiceRW/People?$orderby=UserName

{
    "@odata.context": "http://services.odata.org/V4/(S(ysqt4lcalbsipb1qkoc04ryb))/TripPinServiceRW/$metadata#People",
    "@odata.nextLink": "https://services.odata.org/V4/(S(ysqt4lcalbsipb1qkoc04ryb))/TripPinServiceRW/People?%24orderby=UserName&%24skiptoken=8",
    "value": [
        ... 8 records ...
    ]
}

GET: ~/TripPinServiceRW/People?$orderby=UserName desc

{
    "@odata.context": "http://services.odata.org/V4/(S(ysqt4lcalbsipb1qkoc04ryb))/TripPinServiceRW/$metadata#People",
    "@odata.nextLink": "https://services.odata.org/V4/(S(ysqt4lcalbsipb1qkoc04ryb))/TripPinServiceRW/People?%24orderby=UserName+desc&%24skiptoken=8",
    "value": [
        ... 8 records ...
    ]
}

According to the specification, $skiptoken is an arbitrary token that services can use to retrieve the next page of data from a previously prepared set. The value of the $skiptoken will have special significance to the server itself and it may be an arbitrary token or reference pointing to a page in a cached resultset.

11.2.5.7 Server-Driven Paging
Responses that include only a partial set of the items identified by the request URL MUST contain a link that allows retrieving the next partial set of items. This link is called a next link; its representation is format-specific. The final partial set of items MUST NOT contain a next link.

OData clients MUST treat the URL of the next link as opaque, and MUST NOT append system query options to the URL of a next link. Services may not allow a change of format on requests for subsequent pages using the next link. Clients therefore SHOULD request the same format on subsequent page requests using a compatible Accept header. OData services may use the reserved system query option $skiptoken when building next links. Its content is opaque, service-specific, and must only follow the rules for URL query parts.

It is worth highlighting this very specific note in the specification:

OData clients MUST NOT use the system query option $skiptoken when constructing requests.

The $skiptoken is a server-side implementation and in many cases you couldn't even guess what a correct value might be. The TripPin service is a very simple demonstration API, it uses a page size of 8 to illustrate the behaviour of server-side paging, given the small size of the overall dataset (20) this is a nice arbitrary number that will result in multiple pages with the last page only partially full. That's enough to test basic compliance of server-side supporting data interfaces.

Server-side paging is designed to encourage search driven interfaces where users formulate more precise search criteria in preference to browsing through the pages 1 by 1. Virtual Scrolling is a common user interface paradigm that exploits server-side paging. Different languages and runtimes have different implementations but basically the user can scroll through a list or grid of data and when they get to the bottom there might be a link to "load more" records (behind the scenes, this will load the response from the nextLink). Sometimes this link is not displayed and the data is automatically loaded as the user approaches or reaches the end of the list.

You can still use traditional client-side paging using the $top and $skip query parameters, however some service implementations of server-side paging will still constrain the results to the fixed number of rows as defined by that servers internal logic. If you are implementing client-side paging then you may still need to use the nextLink to retrieve all the results for each client-side page of results.

Lets compare by first getting page 2, of a client-side page size of 5:

GET: ~/TripPinServiceRW/People?$skip=5&$top=5

{
    "@odata.context": "http://services.odata.org/V4/(S(ysqt4lcalbsipb1qkoc04ryb))/TripPinServiceRW/$metadata#People",
    "value": [
        ... 5 records ...
    ]
}

Notice that there is no @odata.nextLink property in the response, that is because the requested number of items does not exceed the server page size logic. So lets try a page size of 9. This time, to retrieve all the records for the page, we will need to make multiple queries.

The general guidance here is to recursively call the service for each of the nextLink urls in the responses, if they contain a nextLink

GET: ~/TripPinServiceRW/People?$skip=9&$top=9

{
    "@odata.context": "http://services.odata.org/V4/(S(ysqt4lcalbsipb1qkoc04ryb))/TripPinServiceRW/$metadata#People",
    "@odata.nextLink": "https://services.odata.org/V4/(S(ysqt4lcalbsipb1qkoc04ryb))/TripPinServiceRW/People?%24skip=9&%24top=9&%24skiptoken=8",
    "value": [
        ... 8 records ...
    ]
}

GET: ~/TripPinServiceRW/People?%24skip=9&%24top=9&%24skiptoken=8

{
    "@odata.context": "http://services.odata.org/V4/(S(ysqt4lcalbsipb1qkoc04ryb))/TripPinServiceRW/$metadata#People",
    "value": [
        ... 1 record ...
    ]
}

When you are implementing your own OData-v4 conformant API it is important to understand this quirk and to document specifically in your API documentation what your policy or convention is with regard to server-side paging and which collections it is enabled on.

In my own implementations, I will often disable server-side paging if the request contains the client-paging token $top (and the value for $top is <= 200) but

From a client-side implementation if you see a nextLink property in the response and you didn't recieve the expected number of records, then you should query for the subsequent server-pages to fulfil your request if you are not able to implement a virtual Scroll enabled user experience.

---

_{NOTE: Normally when we discuss paging in OData v4 services the examples would include use of the $count query option.}

_{[11.2.9 Requesting the Number of Items in a Collection]: On success, the response body MUST contain the exact count of items matching the request after applying any $filter or $search system query options...}

_{The returned count MUST NOT be affected by $top, $skip, $orderby, or $expand.}

_{The TripPin service DOES NOT CONFORM to this particular (and many other) clause in the specification, so I have not used that query option in this explanation.}