Reduce Latency in MIDI GUI

Question

I'm trying to create a simple MIDI display using mido and PySimpleGUI. I have it working decently well, but am hoping to reduce latency between the MIDI controller (i.e. a MIDI keyboard) and the interface display. Particularly, the display will begin to lag once notes are played relatively fast, and then even after I slow down and continue to play at a slower rate. The latency will then only go away if I close out of the GUI and re-launch it. I can't tell exactly if the issue is with mido, PySimpleGUI, or something else in my implementation, but since there isn't any latency in the actual sound coming out, and it appears there's no delay when I use mido in isolation (i.e. just printing notes to a Jupyter notebook), my money is on PySimpleGUI or my inefficient code being the culprit.

For the sake of this post I've tried to reduce my implementation to the simplest terms possible, which is just a script that makes a note being pressed on the MIDI controller trigger a 'c' key being pressed on the computer keyboard using pynput (this is a weird workaround because as far as I can tell you cannot directly trigger a PySimpleGUI event through a MIDI controller), as well as a basic PySimpleGUI interface that displays the pitch value of the note being played.

Below is the MIDI script which I run asynchronously in a separate notebook:

from pynput.keyboard import Key, Controller

def trigger():

    keyboard = Controller()
    key = "c"
        
    try:
        with mido.open_input(name='IAC Driver Mido Test') as port:
            for message in port:
                keyboard.press(key)
                keyboard.release(key)                
    except KeyboardInterrupt:
        pass

And below is the simplified PySimpleGUI setup to read MIDI data:

import PySimpleGUI as sg

with mido.open_input(name='IAC Driver Mido Test') as port:

    # Window Dimensions
    width = 1300
    height = 600

    # Arbitrary 'c' key linked to MIDI controller through pynput
    callbacks = ['c'] 

    canvas = [[sg.Canvas(size=(width, height), background_color='black', key= 'canvas')]]
    
    # Show the Window to the user
    window = sg.Window('MIDI Testing', canvas, size=(width, height), return_keyboard_events=True, use_default_focus=False)

    # Event loop. Read buttons, make callbacks
    while True:
        canvas = window['canvas']
        
        # Initialize note
        note = 0
        for msg in port.iter_pending():
            note_type = msg.type
            if note_type == 'note_on':
                note = msg.note
        
        # Read the Window
        event, value = window.read()

        # If a note is played
        if event in callbacks:        
            if note!=0:
                rect = canvas.TKCanvas.create_rectangle(0, 0, width, height)
                canvas.TKCanvas.itemconfig(rect, fill="Black")
                # Display the pitch value
                canvas.TKCanvas.create_text(width/2, height/2, text=str(note), fill="White", font=('Times', '24', 'bold'))

        # Close the window
        if event in (sg.WIN_CLOSED, 'Quit'):
            break

    window.close()

I've had trouble finding much info out there on this issue as it's pretty niche, but I imagine with all the much more advanced music software out there that have low latency MIDI displays (i.e. Ableton, GarageBand), there might be a better way to go about doing what I'm trying to accomplish here. Any pointers or critiques would be greatly appreciated!

Answer 1

Know nothing about mido, but something maybe wrong here

• keyboard event is not necessary, just call window.write_event_value to generate event.
• Iterate over port.iter_pending each time before you read event may got latency in your event loop.

Here, multithread used to monitor the input from mido and call write_event_value to generate event to event loop to update GUI.

Not sure if port.iter_pending will keep running to monitor the input of mido, so a while loop added to keep it running. A sleep call there maybe help to reduce CPU consumption, of course, there will be a 10ms delay.

Following code not yet executed, maybe failed to run for something missed or wrong.

from time import sleep
import threading
import PySimpleGUI as sg

def mido_thread(window):

    global running

    with mido.open_input(name='IAC Driver Mido Test') as port:
        while running:
            for msg in port.iter_pending():
                note_type = msg.type
                if note_type == 'note_on':
                    note = msg.note
                    if note != 0:
                        window.write_event_value('Note', note)
            sleep(0.01)
    window.write_event_value('Mido End', None)

width, height = size = (1300, 600)
layout = [
    [sg.Graph(size, (0, 0), size, background_color='black', key= '-Graph-')],
    [sg.Push(), sg.Button('Quit')],
]
window = sg.Window('MIDI Testing', layout, finalize=True,  enable_close_attempted_event=True)
graph = window['-Graph']
running, text = True, None
threading.Thread(target=mido_thread, args=(window, ), daemon=True).start()

while True:

    event, value = window.read()

    if event in (sg.WINDOW_CLOSE_ATTEMPTED_EVENT, 'Quit'):
        running = False

    elif event == 'Note':               # Update note from thread
        note = str(values[event])
        if text:
            graph.delete_figure(text)
        text = graph.draw_text(note, (width/2, height/2), color='white', font=('Times', '24', 'bold'))

    elif event == 'Mido End':           # wait thread end
        break

window.close()