Speech to Phoneme in .Net

Question

The problem is that I want to get phonemes of a audio speech in C# language. say you have an audio file like "x.wav" that says "hello dear Shamim". i want to extract all the phonemes of the speech and their relative timings. something like the picture below:

I used System.Speech library (both recognition and synthesis namespaces) but i didn't find what i wanted. Now don't be mistaken! I don't want the phonemes of the sentence "hello dear Shamim", i want to extract the phonemes from an unknown audio input that speaks and English sentence. I tried System.Speech.Recognition but it tries to extract the words out of the audio file, not the phonems! and as you may guessed, the words are 30% wrong! ;)

Answer 1

Phoneme recognition requires a bit of a specialized set-up compared to word recognition, and most engines don't support it directly (a dictionary of monophonic "words" doesn't usually result in good accuracy). A big reason for that is that phoneme recognition is much less accurate than word recognition, since word recognition is more constrained (it filters out all phone combinations which don't map to real words, which is most of them). But HTK does support it. You can use it by executing shell commands (there's nothing evil in doing that from C#) or pinvoking the libraries.

Answer 2

You can bind Hidden Markov Model Tool Kit with pinvoke to your c# code or try to use Accord.net framework, which is managed and has HMM classes, but no concrete methods to extract phonemes.

Answer 3

Is this for vanilla .net, or can you use SAPI (you know, speech API)? The speech api is nice, and it seems to have what you are looking for. Most of all, in a windows environment, it is easily obtained than the external libraries(not to mention that there is not much of a licencing issue regardless of application).

Did you notice System.Speech.Recognition.RecognizedWordUnit? That seems to be roughly what you are looking for.

Answer 4

Try using the System.Speech.Recognition.DictationGrammar constructor that takes a string argument, and pass "grammar:dictation#pronunciation" as the argument. Alternatively, raw SAPI (using the SpeechLib interop assembly) can specify the pronunciation grammar via ISpRecoGrammar::LoadDictation and specifying "Pronunciation" as the dictation topic.