Socket - Address already in use on free socket - BUT only if not in debug mode

Question

I wrote a java program that is listening on a socket.

...
int port = getPort();
ServerSocket server = new ServerSocket(port);
server.accept()
...

It works fine over a decade or so, with Java 1.4, 5 and 6. But with Java 7 or 8 the constructor always fails with the following bind exception:

java.net.BindException: Address already in use: JVM_Bind
    at java.net.DualStackPlainSocketImpl.bind0(Native Method)
    at java.net.DualStackPlainSocketImpl.socketBind(DualStackPlainSocketImpl.java:106)
    at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:382)
    at java.net.PlainSocketImpl.bind(PlainSocketImpl.java:190)
    at java.net.ServerSocket.bind(ServerSocket.java:375)
    at java.net.ServerSocket.<init>(ServerSocket.java:237)
    at java.net.ServerSocket.<init>(ServerSocket.java:128)

I am absolutely sure, the port is free and after a few tests I found out two things:
1. When starting the JVM with the debug options
-Xdebug -Xrunjdwp:server=y,transport=dt_socket,address=8000,suspend=n,
it works!
2. Without setting the debug options, only the first call of the constructor fails, the second (with the same port) is always successfull! So the following ugly code is a workaround:

...
int port = getPort();
try {
     server = new ServerSocket(port);
} catch(BindException e){
    server = new ServerSocket(port);
}
server.setSoTimeout(0);
server.accept()
...
<br>

But I want to use that in no case :)

After i found out that, i have removed the debug options from my tomcat startup file. Surprisingly Tomcat 7 has the same problem to create the HTTP listener when starting with Java 7/8 without the debug options. But Tomcat 7 needs Java 7 or higher.
I am sure that neither Java 7/8 nor Tomcat 7 were shipped out with a bug in such an important thing like socket communication.
So what I am doing wrong ?

I have tested with Windows 7 Professional SP1 64 Bit and JDK 7 32/64 bit and JDK 8 64 bit.

Update;
On another machine, with the same OS and jdk, the problem does not occur.