How to print iterations per second?

Question

I have a small Python script which sends POST requests to a server and gets their response.

It iterates 10000 times, and I managed to print the current progress in command prompt using:

code=current_requestnumber
print('{0}/{1}'.format(str(code),"10000"),end="\r")

at the end of each loop.

Because this involves interaction with a webserver, I would like to show the current average speed next to this too (updated like every 2 seconds).

An example at the bottom of the command prompt would then be like this:

(1245/10000), 6.3 requests/second

How do I achieve this?

Answer 1

You can get a total average number of events per second like this:

#!/usr/bin/env python3

import time
import datetime as dt

start_time = dt.datetime.today().timestamp()
i = 0
while(True):
    time.sleep(0.1)
    time_diff = dt.datetime.today().timestamp() - start_time
    i += 1
    print(i / time_diff)

Which in this example would print approximately 10. Please note that I used a timestamp method of datetime which is only availble in Python 3.

Now, if you would like to calculate the "current" number of events per second, say over the last 10 events, you can do it like this:

#!/usr/bin/env python3

import time
import datetime as dt

last_time = dt.datetime.today().timestamp()
diffs = []
while(True):
    time.sleep(0.1)

    # Add new time diff to list
    new_time = dt.datetime.today().timestamp()
    diffs.append(new_time - last_time)
    last_time = new_time

    # Clip the list
    if len(diffs) > 10:
        diffs = diffs[-10:]

    print(len(diffs) / sum(diffs))

Here, I'm keeping a list of durations of last 10 iterations over which I can then use to get the average number of events per second.

Answer 2

Count finished requests and divide it by the time the program took to execute already?

time.time will be useful for getting the time. The rest is just dividing of current_requestnumber by the differnce in seconds since start... That way you get the long term average. If you need to quickly show when the speed changes it gets a bit more complicated because you need to count the number of requests for say the last 10 seconds. (one way is just reset the counter when more than 10 seconds have passed since the last reset)

Answer 3

You can use the tqdm library for this. A simple example for this is

from tqdm import tqdm
for i in tqdm(range(1e20)):
    ##LOOP BODY

This will print the current iteration, iterations/second, ETA and a nice progress bar

e.g.

 21%|████████████████████  21/100 [01:45<04:27,  3.39s/it]

Answer 4

It depends whether your requests are sent synchronously or asynchronously.

For synchronous sending, tqdm library provides the required statistics.

For asynchronous sending you should wrap your request sending code with a code saving the time it took for each request to be sent, and as a callback update a global object that holds the statistics and updates it to screen. The implementation depends on the library you use for the asynchronous sending.

Answer 5

Using progress bar 2 here is the solution:

import time
import progressbar
widgets = [
    progressbar.Percentage(),
    ' ', progressbar.SimpleProgress(format=f'({progressbar.SimpleProgress.DEFAULT_FORMAT})'),
    ' ', progressbar.Bar(),
    ' ', progressbar.Timer(), ' | '
                              ' ', progressbar.ETA(), ' | ',
    ' ', progressbar.AdaptiveTransferSpeed(unit='it'),
]
bar = progressbar.ProgressBar(widgets=widgets)
for i in bar(range(100)):
    time.sleep(0.1)
    bar.update(i)
100% (100 of 100) |#####| Elapsed Time: 0:00:10 |  Time:  0:00:10 |    9.8 it/s

see discussion for details, acknowledgement to the fantastic forum the maintainers of progressbar 2 have: https://github.com/WoLpH/python-progressbar/discussions/253

Answer 6

With python-progressbar2 you can do it like this:

import time
import progressbar


widgets = [
    progressbar.Percentage(),
    progressbar.Bar(),
    ' ', progressbar.SimpleProgress(),
    ' ', progressbar.ETA(),
    ' ', progressbar.AdaptiveTransferSpeed(unit='it'),
]

bar = progressbar.ProgressBar(widgets=widgets)
for i in bar(range(100)):
    time.sleep(0.2)
    bar.update(i)

This does roughly what you want I think :)

 19%|########                               | 19 of 100 ETA:   0:00:17   4.9 it/s

(disclaimer: I'm the author and maintainer of python-progressbar2)