Unable to iterate over SegmentList while more than one match is found

Question

I'm modifying the pocketsphinx android demo to test continuous keywords spotting based on a keywords list and relative thresholds.

When the onResult method of my implementation of edu.cmu.pocketsphinx.RecognitionListener is called this string hypothesis.getHypstr() will contain the list of possible matches.

I read here that to get every single match and their weights it is possible to do like this:

for (Segment seg : recognizer.getDecoder().seg()) {
    System.out.println(seg.getWord() + " " + seg.getProb());
}

However my code running is never iterating over segments like if SegmentList was empty while hypothesis.getHypstr() shows more than one match.

To reproduce the case I'm using this keyword list with very low tresholds so that more matches are easily found:

rainbow /1e-50/
about /1e-50/
blood /1e-50/
energies /1e-50/

My onPartialResult method is doing nothing while:

public void onEndOfSpeech() {
        switchSearch(KWS_SEARCH);
}

public void onResult(Hypothesis hypothesis) {
    if (hypothesis != null) {

    for (Segment seg : recognizer.getDecoder().seg()) {
        //No iteration is done here!!!
        Log.d("onResult", seg.getWord() + " " + seg.getProb());
    }

        String text = hypothesis.getHypstr();
        makeText(getApplicationContext(), text, Toast.LENGTH_SHORT).show();
    }
}

For example if I say "energies" then hypothesis.getHypstr()="blood about energies blood" but no iteration is done over SegmentList: I can see it by putting a breakpoint at the beginning of the onResult method.

Any suggestion?

Thanks

Answer 1

There is a threading issue here. onResult message is delivered when recognizer is already restarted in switchSearch and so hypothesis is cleared and query for result returns nothing.

You can put this code inside switchSearch before recognizer is restarted, then it will work ok:

private void switchSearch(String searchName) {
    boolean wasRunning = recognizer.stop();

    if (wasRunning) {
        for (Segment seg : recognizer.getDecoder().seg()) {
            Log.d("!!!! ", seg.getWord());
        }
    }

    // If we are not spotting, start listening with timeout (10000 ms or 10 seconds).
    if (searchName.equals(KWS_SEARCH))
        recognizer.startListening(searchName);
    else
        recognizer.startListening(searchName, 10000);

    String caption = getResources().getString(captions.get(searchName));
    ((TextView) findViewById(R.id.caption_text)).setText(caption);
}

If you use only keyword spotting, you can also put this code inside onPartialResult which is invoked as soon as keyphrase is detected, not when silence is detected. That makes reaction faster. You do not need onEndOfSpeech and onResult in pure keyword spotting.