Watson Discovery: Problem with processing data through the API

337 views Asked by At

I'm trying to process 30Mb JSON file to Watson Discovery with the Node SDK, but it gives the error that the file is too large. In the documentation, there is explicitly said that we can process up to 50Mb JSON data with the API.

Where the issue might come from? (The JSON has a root array element and each of the objects contain only two Strings)

UPDATE: The JSON file has the following structure

{
  "elements":[
    {
       "Q":" ... ",
       "A":" ... "
    },
    ...
  ]
}
1

There are 1 answers

1
chughts On

From the API Documentation - https://cloud.ibm.com/apidocs/discovery-data#adddocument

413 : Too large. Returned if you attempt to add a document or document metadata that exceeds the maximum possible.

So either the document or metadata is too large. Your data shouldn't be, so that leaves metadata. Again from the API Documentation -

metadata string The maximum supported metadata file size is 1 MB. Metadata parts larger than 1 MB are rejected.

Example: { "Creator": "Johnny Appleseed", "Subject": "Apples" }