Using a generator with Django's Paginator

Question

Is it possible for Django's Paginator to work with a Python generator?

For example, assuming I have a generator which will automatically paginate requests to an API like this pseudocode:

def get_response():
    page_one = requests.get('http://example.com/json?page=1')
    for item in page_one:
        yield item

    page_two = requests.get('http://example.com/json?page=2')
    for item in page_two:
        yield item

Could Django's Paginator be extended to support this generator?

generator = get_response()
paginator = Paginator(generator, 10)

I know this approach will work:

paginator = Paginator(list(generator), 10)

My understanding is that passing a generator directly would be more efficient because the API client would not make a request until it was required to do so. Though I understand that a Paginator requires a count() or __len__() method which may make this impossible.

Answer 1

You could probably do this, but it would be of limited use.

The part of the Paginator class that slices the list to create a page is at the end of the page method. It does this via slicing of the object list; but obviously, a generator does not support slicing.

So the only thing you could do would be to replace that whole method, and instead of the slicing you would have to repeatedly call next on the generator until you got to the page you wanted. You would gain over calling list() on the whole object list, but the amount of code you'd need to replace probably wouldn't make it worthwhile to use this class at all.