Decoding list containing dictionaries

Question

I need to get certain values out of a list of dictionaries, which looks like that and is assigned to the variable 'segment_values':

[{'distance': 114.6,
  'duration': 20.6,
  'instruction': 'Head north',
  'name': '-',
  'type': 11,
  'way_points': [0, 5]},
 {'distance': 288.1,
  'duration': 28.5,
  'instruction': 'Turn right onto Knaufstraße',
  'name': 'Knaufstraße',
  'type': 1,
  'way_points': [5, 17]},
 {'distance': 3626.0,
  'duration': 273.0,
  'instruction': 'Turn slight right onto B320',
  'name': 'B320',
  'type': 5,
  'way_points': [17, 115]},
 {'distance': 54983.9,
  'duration': 2679.3,
  'instruction': 'Keep right onto Ennstal-Bundesstraße, B320',
  'name': 'Ennstal-Bundesstraße, B320',
  'type': 13,
  'way_points': [115, 675]},
 {'distance': 11065.1,
  'duration': 531.1,
  'instruction': 'Keep left onto Pyhrn Autobahn, A9',
  'name': 'Pyhrn Autobahn, A9',
  'type': 12,
  'way_points': [675, 780]},
 {'distance': 800.7,
  'duration': 64.1,
  'instruction': 'Keep right',
  'name': '-',
  'type': 13,
  'way_points': [780, 804]},
 {'distance': 49.6,
  'duration': 4.0,
  'instruction': 'Keep left',
  'name': '-',
  'type': 12,
  'way_points': [804, 807]},
 {'distance': 102057.2,
  'duration': 4915.0,
  'instruction': 'Keep right',
  'name': '-',
  'type': 13,
  'way_points': [807, 2104]},
 {'distance': 56143.4,
  'duration': 2784.5,
  'instruction': 'Keep left onto S6',
  'name': 'S6',
  'type': 12,
  'way_points': [2104, 2524]},
 {'distance': 7580.6,
  'duration': 389.8,
  'instruction': 'Keep left',
  'name': '-',
  'type': 12,
  'way_points': [2524, 2641]},
 {'distance': 789.0,
  'duration': 63.1,
  'instruction': 'Keep right',
  'name': '-',
  'type': 13,
  'way_points': [2641, 2663]},
 {'distance': 815.9,
  'duration': 65.3,
  'instruction': 'Keep left',
  'name': '-',
  'type': 12,
  'way_points': [2663, 2684]},
 {'distance': 682.9,
  'duration': 54.6,
  'instruction': 'Turn left onto Heinrich-Drimmel-Platz',
  'name': 'Heinrich-Drimmel-Platz',
  'type': 0,
  'way_points': [2684, 2711]},
 {'distance': 988.1,
  'duration': 79.0,
  'instruction': 'Turn left onto Arsenalstraße',
  'name': 'Arsenalstraße',
  'type': 0,
  'way_points': [2711, 2723]},
 {'distance': 11.7,
  'duration': 2.1,
  'instruction': 'Turn left',
  'name': '-',
  'type': 0,
  'way_points': [2723, 2725]},
 {'distance': 0.0,
  'duration': 0.0,
  'instruction': 'Arrive at your destination, on the left',
  'name': '-',
  'type': 10,
  'way_points': [2725, 2725]}]

I need to get the duration values and the waypoint values out of that code segment.

For the duration I tried:

segment_values= data['features'][0]['properties']['segments'][0]['steps'] #gets me the above code
print(segment_values[0:]['duration']) 

Shouldn't this print me all dictionaries, and the values at duration in each of them?

I also tried this:

duration = data['features'][0]['properties']['segments'][0]['steps'][0:]['duration']
print(duration)

Both tries give me "TypeError: list indices must be integers or slices, not str "

Where am I going wrong?

Answer 1

Your data is a list of dictionaries.

For this reason you need to cycle through its content in order to access data. Please try this print statement to look at the data more closely:

for item in data_list:
    print(item)

In order to access duration per each item you can use similar code:

for item in data_list:
    print(item['duration'])

You can also use list comprehension to achieve the same result:

duration = [item['duration'] for item in data_list]

List comprehension is a Pythonic way to obtain the same result, you can read more about it here.

The same principle can be applied twice if a key in your data contains a list or another iterable, here's another example:

for item in data:
    print("\nPrinting waypoints for name: " + item['name'])
    for way_point in item['way_points']:
        print(way_point)

Answer 2

duration = [x['duration'] for x in segment_values]
waypoints =[x['way_points'] for x in segment_values]

Answer 3

You might be thinking of higher-level wrappers like pandas, which would let you do

>>> import pandas as pd
>>> import numpy as np
>>> df = pd.DataFrame(np.random.randn(3, 2), index=list('abc'), columns=list('xy'))
>>> df
          x         y
a -0.192041 -0.312067
b -0.595700  0.339085
c -0.524242  0.946350

>>> df.x
a   -0.192041
b   -0.595700
c   -0.524242
Name: x, dtype: float64

>>> df[0:].x
a   -0.192041
b   -0.595700
c   -0.524242
Name: x, dtype: float64

>>> df[1:].y
b    0.339085
c    0.946350
Name: y, dtype: float64

Answer 4

Another tool for this is glom, which provides helpers for logic like this (pip install glom).

>>> from glom import glom
>>> from pprint import pprint
>>> data = <your data>
>>> pprint(glom(data, [{'wp': 'way_points', 'dist': 'distance'}]))
[{'dist': 114.6, 'wp': [0, 5]},
 {'dist': 288.1, 'wp': [5, 17]},
 {'dist': 3626.0, 'wp': [17, 115]},
 {'dist': 54983.9, 'wp': [115, 675]},
 {'dist': 11065.1, 'wp': [675, 780]},
 {'dist': 800.7, 'wp': [780, 804]},
 {'dist': 49.6, 'wp': [804, 807]},
 {'dist': 102057.2, 'wp': [807, 2104]},
 {'dist': 56143.4, 'wp': [2104, 2524]},
 {'dist': 7580.6, 'wp': [2524, 2641]},
 {'dist': 789.0, 'wp': [2641, 2663]},
 {'dist': 815.9, 'wp': [2663, 2684]},
 {'dist': 682.9, 'wp': [2684, 2711]},
 {'dist': 988.1, 'wp': [2711, 2723]},
 {'dist': 11.7, 'wp': [2723, 2725]},
 {'dist': 0.0, 'wp': [2725, 2725]}]

You can get a feel on how other cases might work from the documentation: https://glom.readthedocs.io/en/latest/faq.html#how-does-glom-work

def glom(target, spec):

    # if the spec is a string or a Path, perform a deep-get on the target
    if isinstance(spec, (basestring, Path)):
        return _get_path(target, spec)

    # if the spec is callable, call it on the target
    elif callable(spec):
        return spec(target)

    # if the spec is a dict, assign the result of
    # the glom on the right to the field key on the left
    elif isinstance(spec, dict):
        ret = {}
        for field, subspec in spec.items():
           ret[field] = glom(target, subspec)
        return ret

    # if the spec is a list, run the spec inside the list on every
    # element in the list and return the new list
    elif isinstance(spec, list):
        subspec = spec[0]
        iterator = _get_iterator(target)
        return [glom(t, subspec) for t in iterator]

    # if the spec is a tuple of specs, chain the specs by running the
    # first spec on the target, then running the second spec on the
    # result of the first, and so on.
    elif isinstance(spec, tuple):
        res = target
        for subspec in spec:
            res = glom(res, subspec)
        return res
    else:
        raise TypeError('expected one of the above types')