loop over mulitple rrd files in directory

Question

I need to iterate through all .rrd files inside a given directory and do fetch data inside each rrd database and do some actions and export them into single csv file in python script!

How can this be done in a efficient way? it's ok to advice me how to loop and access database data over multiple files?

Answer 1

I assume you have a rrdtool installation with python bindings already on your system. If not, here is an installation description.

Then, to loop over .rrd files in a given directory and performing a fetch:

import os
import rrdtool

target_directory = "some/directory/with/rrdfiles"
rrd_files = [os.path.join(target_directory, f) for f in os.listdir(target_directory) if f.endswith('.rrd')]
data_container = []
for rrd_file in rrd_files:
    data = rrdtool.fetch(rrd_file, 'AVERAGE'  # or whichever CF you need
        '--resolution', '200',                # for example
    )                                         # and any other settings you need
    data_container.append(data)

The parameter list follows rrdfetch.

Once you have whatever data you need inside the rrd_files loop, you should accumulate it in a list of lists, with each sublist being one row of data. Writing them to a csv is as easy as this:

import csv

# read the .rrd data as above into a 'data_container' variable

with open('my_data.csv', 'w', newline='') as csv_file:
    rrd_writer = csv.writer(csv_file)
    for row in data_container:
        rrd_writer.writerow(row)

This should outline the general steps you have to follow, you will probably need to adapt them (the rrdfetch in particular) to your case.