Tracking Memory Usage in Python

Question

Python code:

def my_function():
    my_list = [i for i in range(1_000_000)]
    return my_list

my_list = my_function()

Memory usage reported by different libraries:

memory_profiler: 82MB

tracemalloc: 36MB

psutil: 3MB

why there is some much difference in the memory usages calculated by different libraries.

Answer 1

Your function my_function() creates a list with 1.000.000 integers, which will take up a significant amount of memory and the difference in memory it's because different memory profiling libraries use different methods to measure memory usage

memory_profiler uses a line-by-line memory profiler that calculates the memory usage of each line of code.

tracemalloc uses a lower-level approach to measure memory usage. It tracks memory allocations and deallocations directly from the Python interpreter, which can result in lower reported memory usage.

psutil uses system-level APIs to measure memory usage. It reports the memory usage of the entire process, including both Python objects and other system resources used by the process.