Django Rest Framework return empty list in results randomly, but count still correct

Question

When I'm trying to use custom filter it's working all the time in local development, but when we deploy to Production (Docker Swarm) We found an issue that sometime the API response is return empty results randomly, but the count is correct. Below is the example results from the API

API Response

{
    'count': 1, 
    'next': 'http://localhost:8000/api/somethings/?email=test%40example.com&limit=0&offset=0', 
    'previous': None, 
    'results': []
}

Right now we need to restart a uwsgi service (By restarting docker swarm for this service) and the problem is fixed for a moment and randomly happen again.

Here is our DRF view

class SomeView(ListCreateAPIView):
    queryset = SomeModel.objects.all()
    serializer_class = SomeModelSerializer
    filter_backends = (OrderingFilter, DjangoFilterBackend)
    filter_class = CustomFilter
    ordering = ('id',)

    def list(self, request, *args, **kwargs):
        if request.GET.get('all', None):
            # Do something
            serializer = self.get_serializer(queryset, many=True)
            return Response(serializer.data)
        else:
            return super(SomeView, self).list(self, request, *args, **kwargs)

Here is our CustomFilter

from django_filters.rest_framework.filters import CharFilter
import rest_framework_filters as filters


class CustomFilter(filters.FilterSet):
    json_field_separator = '___'

    json_field_is = CharFilter(name='json_field', method='filter_json_field')
    json_field_is_not = CharFilter(name='json_field', method='exclude_json_field')

    def split_lookup_field(self, value):
        return dict(field.split(':') for field in value.split(self.json_field_separator))

    def filter_json_field(self, queryset, name, value):
        try:
            lookup_field = self.split_lookup_field(value)
            return queryset.filter(**lookup_field)
        except (ValueError, FieldError):
            return queryset.none()

    def exclude_json_field(self, queryset, name, value):
        try:
            lookup_field = self.split_lookup_field(value)
        except (ValueError, FieldError):
            return queryset.none()

        for query_arg, query_value in lookup_field.items():
            queryset = queryset.exclude(**{query_arg: query_value})

        return queryset

    class Meta:
        model = SomeModel
        exclude = ['image', 'json_field']

Here is a version of Package we use for this project

Django==1.10.8
djangorestframework==3.6.4
django-filter==1.0.4
djangorestframework-filters==0.10.2

Answer 1

In GenericAPIView you can find a method which is called: get_queryset(), which looks like this:

def get_queryset(self):
    """
    Get the list of items for this view.
    This must be an iterable, and may be a queryset.
    Defaults to using `self.queryset`.

    This method should always be used rather than accessing `self.queryset`
    directly, as `self.queryset` gets evaluated only once, and those results
    are cached for all subsequent requests.

    You may want to override this if you need to provide different
    querysets depending on the incoming request.

    (Eg. return a list of items that is specific to the user)
    """
    assert self.queryset is not None, (
        "'%s' should either include a `queryset` attribute, "
        "or override the `get_queryset()` method."
        % self.__class__.__name__
    )

    queryset = self.queryset
    if isinstance(queryset, QuerySet):
        # Ensure queryset is re-evaluated on each request.
        queryset = queryset.all()
    return queryset

It is a quick call, but I think your queryset is not re-evaluated on each request. Take a look at this comment: # Ensure queryset is re-evaluated on each request.