IBM Watson ToneAnalyzerV3 WatsonException: Error: Invalid JSON input, Code: 400

Question

The following set of code work when I have short text but when I give it a longer text of this form: https://pastebin.com/PLinyN65

import os
import json
from watson_developer_cloud import ToneAnalyzerV3 
from glob import glob

with open(credentials.txt) as cred:
    data = cred.readlines()
    UserName = data[0]
    PassWord = data[1]

tone_analyzer = ToneAnalyzerV3(
                username=UserName,
                password=PassWord,
                version='2016-05-19')

pth = '../input/*.txt'
txtFiles = glob(pth)
for file in txtFiles:
    baseName = os.path.basename(file) 
    name = baseName.split('.')

with open(file) as f:
    txt = f.read()

    with open('../output/{}.json'.format(name[0]), 'w') as outFile:
        json.dump(tone_analyzer.tone(text=txt, sentences='false'), outFile, separators=(',', ': '), indent=2)

I get the error:

WatsonException                           Traceback (most recent call last)
<ipython-input-6-d948e97e307e> in <module>()
      8 
      9         with open('../output/{}.json'.format(name[0]), 'w') as outFile:
---> 10             json.dump(tone_analyzer.tone(text=txt, sentences='false'), outFile, separators=(',', ': '), indent=2)
     11 
     12     print('{}\n'.format(name[0]))

/Library/Python/2.7/site-packages/watson_developer_cloud/tone_analyzer_v3.pyc in tone(self, text, tones, sentences)
     51         data = {'text': text}
     52         return self.request(method='POST', url='/v3/tone', params=params,
---> 53                             json=data, accept_json=True)

/Library/Python/2.7/site-packages/watson_developer_cloud/watson_developer_cloud_service.pyc in request(self, method, url, accept_json, headers, params, json, data, files, **kwargs)
    318             else:
    319                 error_message = self._get_error_message(response)
--> 320             raise WatsonException(error_message)

WatsonException: Error: Invalid JSON input, Code: 400

When I try to manually input the text on this weblink: https://tone-analyzer-demo.mybluemix.net/

I get the error: request entity too large

Please advice how I can get the required tone analysis scores in json format from ToneAnalyzerV3.

Answer 1

Looking at the Tone Analyzer API Explorer, you can see there is a limit on the amount of text each request can handle:

The Tone Analyzer service supports up to 128 KB of text, or about 1000 sentences. Sentences with less than three words cannot be analyzed.

Tone Analyzer API Explorer

The text you provided is 165,743 bytes (168 KB), which is larger than what the service supports. I attempted to input the text provided above in the API Explorer and received the same error as you have above:

{
  "code": 400,
  "sub_code": "C00013",
  "error": "Invalid JSON input"
}

As a solution you will need to use a text input that is below the maximum of 128 KB of text.