why use django haystack slice queryset too slow?

Question

I use whoosh as the search backend.

when I get just 3 search result, code:

#!/usr/bin/python
# -*- coding: utf-8 -*-
import sys
sys.path.append('/home/guomeng/projects/tapplex_ringtones')
import os
os.environ['DJANGO_SETTINGS_MODULE'] = "tapplex_ringtones.settings"

from haystack.query import SearchQuerySet
from ringtones.models import Ringtone
import time

query_word = u'sky'
t0 = time.time()
sqs = SearchQuerySet().models(Ringtone).filter(content=query_word)[:3]
t1 = time.time()
print sqs
print t1 - t0

the result is:

[<SearchResult: ringtones.ringtone (pk=u'1730')>, <SearchResult: ringtones.ringtone (pk=u'28959')>, <SearchResult: ringtones.ringtone (pk=u'25889')>]
0.422543048859> 0.422543048859

when I get all search result, code:

#!/usr/bin/python
# -*- coding: utf-8 -*-
import sys
sys.path.append('/home/guomeng/projects/tapplex_ringtones')
import os
os.environ['DJANGO_SETTINGS_MODULE'] = "tapplex_ringtones.settings"

from haystack.query import SearchQuerySet
from ringtones.models import Ringtone
import time

query_word = u'sky'
t0 = time.time()
sqs = SearchQuerySet().models(Ringtone).filter(content=query_word)
t1 = time.time()
print sqs
print t1 - t0

the result is:

[<SearchResult: ringtones.ringtone (pk=u'1730')>, <SearchResult: ringtones.ringtone (pk=u'28959')>, <SearchResult: ringtones.ringtone (pk=u'25889')>, <SearchResult: ringtones.ringtone (pk=u'5303')>, <SearchResult: ringtones.ringtone (pk=u'5335')>, <SearchResult: ringtones.ringtone (pk=u'5411')>, <SearchResult: ringtones.ringtone (pk=u'1212')>, <SearchResult: ringtones.ringtone (pk=u'28473')>, <SearchResult: ringtones.ringtone (pk=u'23867')>, <SearchResult: ringtones.ringtone (pk=u'27087')>, <SearchResult: ringtones.ringtone (pk=u'26849')>, <SearchResult: ringtones.ringtone (pk=u'2973')>, <SearchResult: ringtones.ringtone (pk=u'2645')>, <SearchResult: ringtones.ringtone (pk=u'31007')>, <SearchResult: ringtones.ringtone (pk=u'11637')>, <SearchResult: ringtones.ringtone (pk=u'16957')>, <SearchResult: ringtones.ringtone (pk=u'106')>, <SearchResult: ringtones.ringtone (pk=u'2481')>, <SearchResult: ringtones.ringtone (pk=u'15697')>]
0.19460105896

why I get all the result is fast?

Answer 1

Probably because:

SearchQuerySet().models(Ringtone).filter(content=query_word)[:3]

Does everything that

SearchQuerySet().models(Ringtone).filter(content=query_word)

does, but also, updates the query set with a limit clause and returns a new queryset. This could account for all or some of the time difference. To get meaningful results you'd need to average the time taken over many calls, and analyze some of the internal code - i.e. how long does the DB take to process the query, and what actually is the query etc.