Python - In-memory size of nested empty lists

Question

The size in memory of an object can be gotten with sys.getsizeof.

As one could expect, the size of [] is smaller than the size of [[]]. On my machine, I get the following sizes:

>>> sys.getsizeof([])
36
>>> sys.getsizeof([[]])
40

Now, whatever the number of nested empty lists I have, I always get the same size:

>>> sys.getsizeof([[[]]])
40
>>> sys.getsizeof([[[[]]]])
40

What is the reason why the size of nested empty lists seems to have an upper boundary?

Answer 1

Reading the documentation would have taught me that when calling getsizeof,

Only the memory consumption directly attributed to the object is accounted for, not the memory consumption of objects it refers to.

Since [] is a container, its size, according to getsizeof, is its own size plus the size of the references it contains, but not the sizes of the objects referred to.

Therefore, if [] has a size of 36, and a reference has a size of 4, then the size of [[]] is 36+4, hence 40.

Now, [[[]]] is nothing more than [x] where x is a reference to [[]]. Hence, the size of [[[]]] is the size of [] plus the size of a reference, so 40 as well.

Answer 2

The size just refers to the outermost object and not the nested ones. From the perspective of getsizeof the object size is just the size of the object plus the size of the pointers contained in the object not the objects being pointed to. You can see this from the following:

>>> import sys
>>> sys.getsizeof([])
64
>>> sys.getsizeof([[]])
72
>>> sys.getsizeof([[[]]])
72
>>> sys.getsizeof([[],[]])
80
>>> sys.getsizeof([[[]],[[]]])
80

If you want to get the total memory footprint you will either need to recursively find the sizes for the object or use some other memory profiling.

Also if you are writing your own objects and want getsizeof to correctly return the size you can implement your own __sizeof__ method. For example:

import sys
class mylist:
    def __init__(self, iterable):
        self.data = list(iterable)

    def __sizeof__(self):
        return object.__sizeof__(self) + \
            sum(sys.getsizeof(v) for v in self.__dict__.values()) + \
            sum(sys.getsizeof(item) for item in self.data)

original_data = [[1,2,3], [1,2,3]]
print(sys.getsizeof(original_data))
foo = mylist(original_data)
print(sys.getsizeof(foo))

Results:

~/code_snippets$ python3 sizeof_list.py 
80
336