I have django project with django-haystack and xapian backend. I have created an index:
sugar=indexes.DecimalField(null=True)
Now I'm trying to filter the SearchQuery with different arithmetical operators, and I'm getting strange results (trying lt, lte, gt and gte with filter like: sqs.filter(sugar__lt=60)). I'm then filtering out None's, sorting the results and converting to float to get a better view on it.
sqs.count()=130, len(sqs_lt)=114, len(sqs_gt)=130, len(sqs_lte)=0, len(sqs_gte)=16
vals_lt=[0.1, 0.1, 0.3, 0.3, 0.4, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.6, 0.6, 0.7, 0.8, 0.8, 0.8, 0.8, 1.0, 1.1, 1.1, 1.1, 1.1, 1.1, 1.3, 1.33, 1.4, 1.5, 1.8, 1.8, 2.0, 2.0, 2.0, 2.0, 2.1, 2.2, 2.2, 2.4, 2.5, 2.5, 2.78, 2.8, 2.9, 3.0, 3.07, 3.2, 3.2, 3.4, 3.6, 3.9, 4.0, 4.9, 6.0, 6.0, 6.2, 6.3, 6.3, 6.6, 12.0, 12.0, 12.4, 12.5, 13.7, 14.0, 18.0, 20.0, 22.0, 22.0, 23.0, 24.1, 24.8, 26.0, 26.0, 26.5, 27.7, 30.96, 34.0, 39.0, 40.0, 40.0, 41.0, 42.2, 42.5, 43.0, 46.0, 48.0, 51.1, 52.0, 54.0, 100.0]
vals_gt=[0.1, 0.1, 0.3, 0.3, 0.4, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.6, 0.6, 0.7, 0.8, 0.8, 0.8, 0.8, 1.0, 1.1, 1.1, 1.1, 1.1, 1.1, 1.3, 1.33, 1.4, 1.5, 1.8, 1.8, 2.0, 2.0, 2.0, 2.0, 2.1, 2.2, 2.2, 2.4, 2.5, 2.5, 2.78, 2.8, 2.9, 3.0, 3.07, 3.2, 3.2, 3.4, 3.6, 3.9, 4.0, 4.9, 6.0, 6.0, 6.2, 6.3, 6.3, 6.6, 7.25, 7.5, 8.2, 8.5, 8.5, 9.3, 12.0, 12.0, 12.4, 12.5, 13.7, 14.0, 18.0, 20.0, 22.0, 22.0, 23.0, 24.1, 24.8, 26.0, 26.0, 26.5, 27.7, 30.96, 34.0, 39.0, 40.0, 40.0, 41.0, 42.2, 42.5, 43.0, 46.0, 48.0, 51.1, 52.0, 54.0, 60.0, 60.0, 63.0, 67.0, 68.0, 70.0, 71.0, 72.0, 72.0, 90.0, 100.0]
vals_lte=[]
vals_gte=[7.25, 7.5, 8.2, 8.5, 8.5, 9.3, 60.0, 60.0, 63.0, 67.0, 68.0, 70.0, 71.0, 72.0, 72.0, 90.0]
I suspect that __lt works almost correctly (except for inexplicable 100), but __gte looks more like lexicographical comparison. What am I missing?
I've taken a deeper look. With the current implementation in xapian_backend, Decimal will be converted to string, hence the lexicographical comparison. I don't know how often people use xapian backend (I've chosen it to have something fast to test MVP), but django-oscar's price in the catalogue.product is Decimal as well, and they're optimistic having it as a DecimalIndex for haystack. For the prices under 10 coins it will mostly work though:)
I'll follow up with gory details of float in the same setup.