using watson discovery to access publically accessible URL

927 views Asked by At

we are using watson discovery service to find answer for travel related questions. Unfortunately the documents set to be used here are not static but the travel related forum on the web. So we need Discovery to access those URL for annotating various components (entities, relations, sentiments etc) and later we can query based on the same to find the right link which has the answers. However I see discovery only support files as documents set and not an URL. I remember alchemy API and the new service NLU both has support for URL. Is there any way discovery can access the URL ? We have selected discovery service for our solution because of the query support it has which seems to be not there in NLU or Alchemy.

1

There are 1 answers

1
Simon O'Doherty On

In order to access a public URL, you would need to create some kind of web crawler which converts the web site to the correct format that the data crawler can access.

More details here: https://www.ibm.com/watson/developercloud/doc/discovery/data-crawler-seeds.html