best approach to automatically load next/prevous page of scraped web page

Question

Situation: I make code to scrape web page. It UWP application based on template 10 and using MVVM and HTML Agility Pack. A scraped page is a list of the result (pages) I can move forward, back in the result list on source web page. All is scraped i.e. a list of web result and web commands to handle such list.

Request: I want to remove buttons "previous page" and "next page". An idea needs to send automatically request to load forward a page or backward page in case I move to top or bottom currently loaded result list (web page).

Notice: I am able to display a message when I am on the top or bottom of the list in View. I have a feeling there is a better way to load previous and next page of the list in ViewModel.

I will be grateful for any suggestion how to do it. All roads lead to Rome, please show me the best way. Thank for help in advance.

Answer 1

I am able to display a message when I am on the top or bottom of the list in View.

If you could get the message when the list is on the top or bottom, you could define a global variable as the requested address and declare a variable as the page index. For example like this: http://www.test.com?page={0}

Then, when you got the "top" or "bottom" message, you could send your HTTP request to that address and get it returned content.

private int pageIndex = 0;
private string address = @"http://www.test.com?page={0}";

private async void GetPreviousPageContent()
{
    pageIndex--;
    Uri uri = new Uri(string.Format(address,pageIndex));
    HttpClient client = new HttpClient();
    var response = await client.GetAsync(uri);
}

private async void GetNextPageContent()
{
    pageIndex++;
    Uri uri = new Uri(string.Format(address, pageIndex));
    HttpClient client = new HttpClient();
    var response = await client.GetAsync(uri);
}

Here, I have another suggestion for you. You could implement two functions for your listview. The Pull to refresh and the ISupportIncrementalLoading. When the listview scrolls to the top and you pull it down, it will fire a specific event, in this event, you could send the HTTP request to get the previous page's content. When the listview scrolls to the bottom, the LoadMoreItemsAsync method of ISupportIncrementalLoading will be fired, in this method, you could get the next page's content. For this way, you would still need to declare two variables as the request address and page index.