How can I change the audio device for a WebRTC publishing stream while running an AudioContext script processor?

Question

For the purposes of this example, I am using the basic publish HTML/js code from an Ant Media Server application (see https://github.com/ant-media/StreamApp/blob/master/src/main/webapp/index.html). (Note - I'm actually using the enterprise edition with WebRTC)

On top of that code, I am creating a script processor to run on an AudioContext in order to display a VU Meter for the microphone levels (code simplified for posting):

function startVUMeters() {
  navigator.mediaDevices.getUserMedia({audio: true, video: true}).then(function (stream) {
    window.vuMeterAudioContext = new AudioContext();
    window.vuMeterAnalyser = vuMeterAudioContext.createAnalyser();
    window.vuMeterMicrophone = vuMeterAudioContext.createMediaStreamSource(stream);
    window.vuMeterAudioScriptProcessor = vuMeterAudioContext.createScriptProcessor(2048, 1, 1);
  
    window.vuMeterMicrophone.connect(vuMeterAnalyser);
    window.vuMeterAnalyser.connect(vuMeterAudioScriptProcessor);
    window.vuMeterAudioScriptProcessor.connect(vuMeterAudioContext.destination);
    window.vuMeterAudioScriptProcessor.onaudioprocess = function () {
        // Calculate the microphone levels and update the meter
    };
  });
}

function stopVUMeters() {
  window.vuMeterMicrophone.disconnect();
  window.vuMeterAnalyser.disconnect();
  window.vuMeterAudioScriptProcessor.disconnect();
  window.vuMeterAudioContext.close();
  window.vuMeterMicrophone = null;
  window.vuMeterAnalyser = null;
  window.vuMeterAudioScriptProcessor = null;
  window.vuMeterAudioContext = null;
}

When the user tries to switch device, I run stopVUMeters() before trying to switch the audio device, which I do by calling switchAudioInputSource() from the ant_webtc_adaptor.js file:

this.switchAudioInputSource = function(streamId, deviceId) {
    //stop the track because in some android devices need to close the current camera stream
    var audioTrack = thiz.localStream.getAudioTracks()[0];
    if (audioTrack) {
        audioTrack.stop();
    }
    else {
       console.warn("There is no audio track in local stream");
    }

    if (typeof deviceId != "undefined" ) {
        thiz.mediaConstraints.audio = { "deviceId": deviceId };
    }
    thiz.setAudioInputSource(streamId, thiz.mediaConstraints, null, true, deviceId);
}

However when I do the switch, it fails telling me that another device is using the microphone.

If I don't start the VU meters at all, then it works fine, so I believe that the issue is with my AudioContext not releasing it's hold on the audio, but I don't know how to solve it. I need to support Safari on iOS, so I can't use audioworklets (yet).

How can I cleanly release the audio so that I can switch audio devices? Alternatively, is there a way I can get the microphone activity from the WebRTC stream without needing to create a new AudioContext in the first place?

Edit

I have been experimenting more. This issue happens in Firefox (Mac). On Chrome (Mac), I can switch devices, but the Audiocontext stays on the default (I think) device (or the device switching isn't working properly, I'm going to check that out too). In Safari it just works. I haven't yet been able to try it on PCs.