SourceDataLine delay to much for real time playing (synth)

Question

I tried making a synth and it works and I can play music with them. However the first synth that I made had delay and you couldn't play fast songs. So I tried again using sourceDataline.flush() method to speed it up. Well it somewhat fixes it but delay is to much. I tried also reducing sample rate but delay is to much.

Edit: turns out you can comment the line keyStateInterface.setFlush(false); it improves the delay however you still can't play fast songs

here is the code:

import javax.sound.sampled.AudioFormat;
import javax.sound.sampled.AudioSystem;
import javax.sound.sampled.LineUnavailableException;
import javax.sound.sampled.SourceDataLine;

public class SoundLine implements Runnable{
    KeyStateInterface keyStateInterface;
    public SoundLine(KeyStateInterface arg){
        keyStateInterface=arg;
    }

    @Override
    public void run() {
        AudioFormat audioFormat = new AudioFormat(44100,8,1,true,false);
        try {
            SourceDataLine sourceDataLine = AudioSystem.getSourceDataLine(audioFormat);
            sourceDataLine.open(audioFormat);
            sourceDataLine.start();

            SynthMain synthMain = new SynthMain();

            int v = 0;
            while (true) {
                int bytesAvailable = sourceDataLine.available();

                if (bytesAvailable > 0) {
                    int sampling = 256/(64);
                    byte[] bytes = new byte[sampling];

                    for (int i = 0; i < sampling; i++) {

                        //bytes[i] = (byte) (Math.sin(angle) * 127f);
                        float t = (float) (synthMain.makeSound((double)v,44100,keyStateInterface)* 127f);
                        bytes[i] = (byte) (t);
                        v += 1;
                    }
                    if(keyStateInterface.getFlush()){
                        sourceDataLine.flush();
                    }
                    sourceDataLine.write(bytes, 0, sampling);
                    //if(!keyStateInterface.isCacheKeysSame())sourceDataLine.flush();

                    //System.out.println(bytesWritten);
                } else {
                    Thread.sleep(1);
                }

                //System.out.println(bytesAvailable);
                //System.out.println();
                //if((System.currentTimeMillis()-mil)%50==0)freq+=0.5;
            }
        }catch (Exception e){


        }
    }
}

public class SynthMain {
    double[] noteFrequency = {
            466.1637615181,
            493.8833012561,
            523.2511306012,
            554.3652619537,
            587.3295358348,
            622.2539674442,
            659.2551138257,
            698.4564628660,
            739.9888454233,
            783.9908719635,
            830.6093951599,
            880.0000000000,
            932.3275230362,
            987.7666025122,
            1046.5022612024,
            1108.7305239075,
            1174.6590716696,
            1244.5079348883,
            1318.5102276515,
            1396.9129257320,
            1479.9776908465,
            1567.9817439270,
            1661.2187903198,
            1760.0000000000,
            1864.6550460724,
            1975.5332050245,
            2093.0045224048,
            2217.4610478150,
            2349.3181433393,
            2489.0158697766,
            2637.0204553030,
            2793.8258514640,
            2959.9553816931,
            3135.9634878540,
            3322.4375806396,
            3520.0000000000,
            3729.3100921447,
    };
    boolean[] keys = new boolean[noteFrequency.length];
    public double makeSound(double dTime,double SampleRate,KeyStateInterface keyStateInterface){
        if(keyStateInterface.getSizeOfMidiKey()>0){
            keyStateInterface.setFlush(true);
            for(int i=0;i<keyStateInterface.getSizeOfMidiKey();i++) {
                KeyRequest keyRequest = keyStateInterface.popMidiKey();
                if(keyRequest.getCommand()==-112){
                    if(keyRequest.getVelocity()>0)keys[keyRequest.getArg1()] = true;
                    if(keyRequest.getVelocity()<1)keys[keyRequest.getArg1()] = false;
                    System.out.println(keyRequest.getVelocity());
                }
            }
        }else{
            keyStateInterface.setFlush(false);
        }
        //System.out.println("makeSound");
        double a = 0.0;
        for(int i=0;i<keys.length;i++){
            if(keys[i]){
                a+=Oscillate(dTime,noteFrequency[i],(int)SampleRate);
            }
        }
        return a*0.4;
    }
    public double Oscillate(double dTime,double dFreq,int sampleRate){
        double period = (double)sampleRate / dFreq;
        return Math.sin(2.0 * Math.PI * (int)dTime / period);
    }
}

import java.util.ArrayList;
import java.util.Stack;

public class KeyState implements KeyStateInterface{
    boolean isFlush;
    ArrayList<KeyRequest> keyRequest = new ArrayList<KeyRequest>();
    ArrayList<KeyRequest> midiKeyRequest = new ArrayList<KeyRequest>();

    @Override
    public void pushKey(int keyCode, boolean press) {
        keyRequest.add(new KeyRequest(KeyRequest.KEY,keyCode,press));
    }

    @Override
    public void pushMidiKey(int command, int arg1, int velocity) {
        midiKeyRequest.add(new KeyRequest(KeyRequest.MIDI_KEY,command,arg1,velocity));
    }

    @Override
    public KeyRequest popKey() {
        KeyRequest t = keyRequest.get(keyRequest.size());
        return t;
    }

    @Override
    public KeyRequest popMidiKey() {
        KeyRequest t = midiKeyRequest.get(keyRequest.size());
        midiKeyRequest.remove(keyRequest.size());
        return t;
    }

    @Override
    public int getSizeOfKey() {
        return keyRequest.size();
    }

    @Override
    public int getSizeOfMidiKey() {
        return midiKeyRequest.size();
    }

    @Override
    public boolean getFlush() {
        boolean v = isFlush;
        isFlush = false;
        return v;
    }

    @Override
    public void setFlush(boolean arg) {
        isFlush=arg;
    }
}

Answer 1

I haven't dug deep into your code, but perhaps the following info will be useful.

The SourceDataLine.write() method uses a blocking queue internally. It will only progress as fast as the data can be processed. So, there is no need to test for available capacity before populating and shipping bytes.

I'd give the SDL thread a priority of 10, since most of it's time is spent in a blocked state anyway.

Also, I'd leave the line open and running. I first got that advice from Neil Smith of Praxis Live. There is a cost associated with continually rebuilding it. And it looks to me like you are creating a new SDL for every 4 bytes of audio data. That would be highly inefficient. I suspect that shipping somewhere in the range of 256 to 8K on a line that is left open would be a better choice, but I don't have hard facts to back that up that opinion. Neil wrote about having all the transporting arrays be the same size (e.g., the array of data produced by the synth be the same size as the SDL write).

I've made a real-time theremin with java, where the latency includes the task of reading the mouse click and position, then sending that to the synth that is generating the audio data. I wouldn't claim thay my latency down to a precision that allows "in the pocket" starts and stops to notes, but it still is pretty good. I suspect further optimization possible on my end.

I think Neil (mentioned earlier) has had better results. He's spoken of achieving latencies in the range of 5 milliseconds and less, as far back as 2011.