fastest async sns publish

Question

i'm trying to send a bunch of messages to a sns asynchronously. i don't want to wait for the sns publish response. i want to send and forget. i'm trying to do the same thing as the sleepy-bois solution here where each sleepy-bois effectively runs in parallel rather than sequence.

if i comment out all the sns publish stuff and just let send_item() sleep. i get: Process time difference: 11.07% and Absolute time difference: 103.85% which is pretty good. but when i include the async sns publish call, those values skyrocket to Process time difference: 1138.72% and Absolute time difference: 1145.18%

is there a better way to to do the asynchronous sns publish to get that absolute difference value closer back to 100%? i feel like it isn't working

import asyncio
import time
from aiobotocore.session import get_session

sns_session = get_session()

async def send_item(destination, message, delay_ms):
    delay_seconds = delay_ms / 1000
    await asyncio.sleep(delay_seconds)
    async with sns_session.create_client('sns') as sns_client:
        await sns_client.publish(
            TopicArn=destination,
            Message=message,
        )

async def send_items(my_items_list):
    await asyncio.gather(*[send_item(item["destination"], item["message"], item["waitTime_ms"]) for item in my_items_list])
    print("Iterated through items.")
    return my_items_list[-1]["waitTime"]

if __name__ == "__main__":
    my_list = get_list()
    start_process_time_ns = time.process_time_ns()
    time_before_send_ns = time.perf_counter_ns()
    last_wait_time_ms = asyncio.run(send_items(my_list))
    time_after__send_ns = time.perf_counter_ns()
    end_process_time_ns = time.process_time_ns()
    total_time_ns = time_after__send_ns - time_before_send_ns
    process_time_ns = end_process_time_ns - start_process_time_ns
    expected_time_ns = last_wait_time_ms * 1000000
    print(f"Expected time (ns): {expected_time_ns}")
    print(f"Process time (ns): {process_time_ns}")
    print(f"Absolute time (ns): {total_time_ns}")
    print(f"Process time difference: {(process_time_ns/expected_time_ns)*100:.2f}%")
    print(f"Absolute time difference: {(total_time_ns/expected_time_ns)*100:.2f}%")
    print("Complete")

Answer 1

Your code is technically not problematic.

In the first case you're not submitting any data, so of course your time divided by a constant will be much shorter than when you publish data and divide a longer time by that constant.

Eventually the difference stems from the time it takes to publish to the SNS.

The code will run asynchronously. If you want, you can check how long does the longest publish take, and you will see that publishing all of messages will amount to a little over that.