Function in Watson conversation

Question

I am successfully using Watson conversation Service. But now requirement is to put math Equation in Watson Response. How do I do that? Also I tried using an external Mathematics keyboard to type, but since my TTS service does not understand equations, I don’t get my desired response. Is there a way to use LaTex here? Also is there a way to put function in my JSON Watson Response. Thanks so much Eg:

Answer 1

I see two questions here:

1. How to send an mathematical equation in a Watson Conversation Service response
2. How to synthesize a mathematical formula to text with the Watson Text to Speech service.

For #1, see my answer at https://stackoverflow.com/a/41489073/933879 - it's basically any text you want including JSON and HTML, and you can optionally convert/translate it before speaking or displaying it to your user.

For #2, you will need to write out the equation in words. So, instead of "f(x)", you'd need to send the TTS service the string "f of x", etc. (The service supports some customization, but I don't think it would work for this situation.)

If you use JSON for the response, you could include both versions - a HTML one for display and a "wordy" one for TTS. (Note that because the service already sends JSON back with a string response, you're going to end up with JSON-inside-of-JSON. Not ideal, but not the end of the world.)

To get started, it's going to be much simpler to do it this way and just write things twice. Your initial "translation" layer will decode the JSON, split those two up and e.g. display the math part as HTML and call the Text to Speech service with the wordy part.

You can see some example apps at https://github.com/watson-developer-cloud?utf8=%E2%9C%93&q=conversation&type=&language= - most of those convert the responses to HTML, so it would just be a simple jump from there to HTML + Audio.

Once you have it working smoothly with manual conversion from formulas to words, you can start working on a system to automate this. I don't have any experience here, but I have a feeling that it's more complex than it sounds. I'm not aware of any existing code to do this, so you're pretty much on your own at this point.

Here's a picture of what I mean when I say JSON for the html and wordy response:

And, here's an example of processing that with a translation layer:

// this is meant to run in a browser
// `WatsonSpeech` is from https://github.com/watson-developer-cloud/speech-javascript-sdk/releases
// `ttsToken` is an auth token you generate server-side, eg: https://github.com/watson-developer-cloud/speech-javascript-sdk/blob/master/examples/tts-token.js
// `res` is the JSON Object returned from the Conversation service's send endpoint - http://www.ibm.com/watson/developercloud/conversation/api/v1/#send_message

function translationLayer(res) {
  var formula = JSON.parse(res.output.text[0]);
  $('#output').append(formula.html);
  WatsonSpeech.TextToSpeech.synthesize({
    text: formula.wordy,
    token: ttsToken
  });
}