Ideal Audio Formatting For Lagless Streaming?

Question

We're currently writing VoIP code, and we're having lag issues, and we've noticed that it changes if we modify a few variables. We're new to working with audio, and we're curious as to why we're having this problem. I've posted code below.

Audio Class:

public class Audio
{
public static final int bufferSize = 320;

    public static AudioFormat getAudioFormat()
    {
    float sampleRate = 16000.0F;
    //8000,11025,16000,22050,44100,48000

    int sampleSizeInBits = 16;
    //8,16

    int channels = 1;
    //1,2

    boolean signed = true;
    //true,false

    boolean bigEndian = false;
    //true,false

    return new AudioFormat(sampleRate, sampleSizeInBits, channels, signed, bigEndian);
    }
}

Recorder Class:

public class Recorder implements Runnable
{
    Socket socket;

    public Recorder(Socket socket)
    {
        this.socket = socket;
    }

    public void run()
    {
        try
        {
            AudioFormat format = Audio.getAudioFormat();

            DataLine.Info dataLineInfo = new DataLine.Info(TargetDataLine.class, format);
            TargetDataLine targetDataLine = (TargetDataLine) AudioSystem.getLine(dataLineInfo);
            targetDataLine.open(format);
            targetDataLine.start();
            AudioInputStream ais = new AudioInputStream(targetDataLine);

            BufferedOutputStream bos = new BufferedOutputStream(socket.getOutputStream());
            Pcm2SpeexAudioInputStream sin = new Pcm2SpeexAudioInputStream(0, 4, ais, format, -1);

            byte[] input = new byte[Audio.bufferSize];
            int count = 0;
            while ((count = sin.read(input)) != -1)
            {
                bos.write(input, 0, count);
            }

            sin.close();
            bos.close();
            socket.close();
        }
        catch (Exception e)
        {
            e.printStackTrace();
        }
    }
}    

We're using JSpeex for our encoding. If anyone knows how to fine tune our variables, please let us know! 4 second lag isn't acceptable, and 10 second definitely isn't.