Django-filter with DRF - How to do 'and' when applying multiple values with the same lookup?

Question

This is a slightly simplified example of the filterset I'm using, which I'm using with the DjangoFilterBackend for Django Rest Framework. I'd like to be able to send a request to /api/bookmarks/?title__contains=word1&title__contains=word2 and have results returned that contain both words, but currently it ignores the first parameter and only filters for word2.

Any help would be very appreciated!

class BookmarkFilter(django_filters.FilterSet):

    class Meta:
        model = Bookmark
        fields = {
            'title': ['startswith', 'endswith', 'contains', 'exact', 'istartswith', 'iendswith', 'icontains', 'iexact'],
        }

class BookmarkViewSet(viewsets.ModelViewSet):
    serializer_class = BookmarkSerializer
    permission_classes = (IsAuthenticated,)
    filter_backends = (DjangoFilterBackend,)
    filter_class = BookmarkFilter
    ordering_fields = ('title', 'date', 'modified')
    ordering = '-modified'
    page_size = 10

Answer 1

The main problem is that you need a filter that understands how to operate on multiple values. There are basically two options:

• Use MultipleChoiceFilter (not recommended for this instance)
• Write a custom filter class

Using MultipleChoiceFilter

class BookmarkFilter(django_filters.FilterSet):
    title__contains = django_filters.MultipleChoiceFilter(
        name='title',
        lookup_expr='contains',
        conjoined=True,  # uses AND instead of OR
        choices=[???],
    )

    class Meta:
        ...

While this retains your desired syntax, the problem is that you have to construct a list of choices. I'm not sure if you can simplify/reduce the possible choices, but off the cuff it seems like you would need to fetch all titles from the database, split the titles into distinct words, then create a set to remove duplicates. This seems like it would be expensive/slow depending on how many records you have.

Custom Filter

Alternatively, you can create a custom filter class - something like the following:

class MultiValueCharFilter(filters.BaseCSVFilter, filters.CharFilter):
    def filter(self, qs, value):
        # value is either a list or an 'empty' value
        values = value or []

        for value in values:
            qs = super(MultiValueCharFilter, self).filter(qs, value)

        return qs


class BookmarkFilter(django_filters.FilterSet):
    title__contains = MultiValueCharFilter(name='title', lookup_expr='contains')

    class Meta:
        ...

Usage (notice that the values are comma-separated):

GET /api/bookmarks/?title__contains=word1,word2

Result:

qs.filter(title__contains='word1').filter(title__contains='word2')

The syntax is changed a bit, but the CSV-based filter doesn't need to construct an unnecessary set of choices.

Note that it isn't really possible to support the ?title__contains=word1&title__contains=word2 syntax as the widget can't render a suitable html input. You would either need to use SelectMultiple (which again, requires choices), or use javascript on the client to add/remove additional text inputs with the same name attribute.

---

Without going into too much detail, filters and filtersets are just an extension of Django's forms.

• A Filter has a form Field, which in turn has a Widget.
• A FilterSet is composed of Filters.
• A FilterSet generates an inner form based on its filters' fields.

Responsibilities of each filter component:

• The widget retrieves the raw value from the data QueryDict.
• The field validates the raw value.
• The filter constructs the filter() call to the queryset, using the validated value.

In order to apply multiple values for the same filter, you would need a filter, field, and widget that understand how to operate on multiple values.

---

The custom filter achieves this by mixing in BaseCSVFilter, which in turn mixes in a "comma-separation => list" functionality into the composed field and widget classes.

I'd recommend looking at the source code for the CSV mixins, but in short:

• The widget splits the incoming value into a list of values.
• The field validates the entire list of values by validating individual values on the 'main' field class (such as CharField or IntegerField). The field also derives the mixed in widget.
• The filter simply derives the mixed in field class.

The CSV filter was intended to be used with in and range lookups, which accept a list of values. In this case, contains expects a single value. The filter() method fixes this by iterating over the values and chaining together individual filter calls.

Answer 2

You can create custom list field something like this:

from django.forms.widgets import SelectMultiple
from django import forms

class ListField(forms.Field):
    widget = SelectMultiple

    def __init__(self, field, *args, **kwargs):
        super(ListField, self).__init__( *args, **kwargs)
        self.field = field

    def validate(self, value):
        super(ListField, self).validate(value)
        for val in value:
            self.field.validate(val)

    def run_validators(self, value):
        for val in value:
            self.field.run_validators(val)

    def to_python(self, value):
        if not value:
            return []
        elif not isinstance(value, (list, tuple)):
            raise ValidationError(self.error_messages['invalid_list'], code='invalid_list')
        return [self.field.to_python(val) for val in value]

and create custom filter using MultipleChoiceFilter:

class ContainsListFilter(django_filters.MultipleChoiceFilter):
    field_class = ListField

    def get_filter_predicate(self, v):
        name = '%s__contains' % self.name
        try:
            return {name: getattr(v, self.field.to_field_name)}
        except (AttributeError, TypeError):
            return {name: v}

After that you can create FilterSet with your custom filter:

from django.forms import CharField

class StorageLocationFilter(django_filters.FilterSet):
    title_contains = ContainsListFilter(field=CharField())

Working for me. Hope it will be useful for you.

Answer 3

Here is a sample code that just works: it supports - product?name=p1,p2,p3 and will return products with name (p1,p2,p3)

def resolve_csvfilter(queryset, name, value):
    lookup = { f'{name}__in': value.split(",") }
    queryset = queryset.filter(**lookup)
    return queryset

class ProductFilterSet(FilterSet):
        name = CharFilter(method=resolve_csvfilter)
    
        class Meta:
            model = Product
            fields = ['name']

Ref: https://django-filter.readthedocs.io/en/master/guide/usage.html#customize-filtering-with-filter-method https://github.com/carltongibson/django-filter/issues/137

Answer 4

For the ones who want to use OR / IN instead of AND:

from django_filters import MultipleChoiceFilter
from django_filters.fields import MultipleChoiceField


class MultipleCharField(MultipleChoiceField):
    def validate(self, _):
        pass


class MultipleCharFilter(MultipleChoiceFilter):
    field_class = MultipleCharField

Usage:

class BookmarkFilter(FilterSet):
    title = MultipleCharFilter(field_name="title", lookup_expr="contains")

Request:

/api/bookmarks/?title=Title1&title=Title2

Check this issue for more detail: GitHub