How Can I Access All of The Keys in a JSON File Using IJSON kvitems?

Question

I'm using ijson.kvitems to iterate over all of the keys in a JSON file that I have.

the JSON file looks like this:

{"filename":{"file_data":
{"name":"samplefile",
"filetype":"Miscellaneous",
"id":123,
"timestamp":"2020-10-08 00:20:00"}}}

based on this answer, a simplified version of my code looks something like so (v is a dictionary too):

import ijson

f = open('file.json')
for k, v in ijson.kvitems(f, ''):
    name = v['name']
    user_id = v['id']
    filetype = v['filetype']
    timestamp = v['timestamp']

I am only able to stream/read about 94% of the keys from the original file this way, trying to figure out if there is a way to get to the remaining 6%.

Thanks!!

Answer 1

The documentation for kvitems maybe isn't fully clear: it returns key/value pairs at the given prefix, and it's not recursive. With your example document and code this is what kvitems returns (note that as of writing ijson.dump isn't yet on the latest PyPI ijson release, but is available on the latest master version on GitHub):

echo '{
  "filename": {
    "file_data": {
      "name":"samplefile",
      "filetype":"Miscellaneous",
      "id":123,
      "timestamp":"2020-10-08 00:20:00"
    }
  }
}' | python -m ijson.dump -m kvitems

#: key, value
-------------
0: filename, {'file_data': {'name': 'samplefile', 'filetype': 'Miscellaneous', 'id': 123, 'timestamp': '2020-10-08 00:20:00'}}

Here key is filename, while value is the rest of the object, since that whole object is the value under filename. In particular keys like name or filetype will not be reported separately; if you wanted those (and their respective values) to be reported you'd have to use a filename.file_data prefix instead.

From the comments in the original question I'm guessing this is the actual problem, but couldn't add this more extensive comment here to further clarify things, and with the hopes it's also the actual answer to your problem.