How do I write my own desktop sharing application in java?

Question

Hello I want write my own desktop sharing application in Java.

The application should have some very default features:

• Capture screen;

• Allow a remote connected user to click / edit fields.

I was thinking to use Java Robot class for mouse movements / key pressing. The problem is i don't know what screen capture strategy to use.

Should I make sequentially screen captures (on the hosting computer) every second, and send those captures with UDP via network, so that the clients can intercept the data-grams ? Isn't this a little overkill for the network ?

What other strategies are available ? (Except trying an already existing app).

PS: If necessary I can even write native code using JNI (still that's the last thing I planning to do).

Later edit: After some investigation I've come to the conclusion of @Thorbjørn Ravn Andersen . Java is probably not the best choice for this kind of application. I can try to use JNI, but that code will cover 75%+ of my project.

I will try to find other alternatives.

Answer 1

Have a good long look at the Ultra VNC project on SourceForge. Great place to start.

Answer 2

In pure Java you cannot access structured information on system windows, nor monitor all relevant system events, so the performance of the display synchronization will not be optimal. Also there are privileged windows, which do not accept mouse or key events from Robot. Remote video streaming is not an option!

With named restrictions your attempt with the Robot class is valid. Robot.createScreenCapture(Rectangle) will put a Desktop section into a BufferedImage, which you can send to the client window. On client side you can capture keyboard and mouse events and send them to a Robot object on the server side.

Without knowing the actual extent of system windows, it will makes sense to work on a grid of Desktop tiles:

• captured BufferedImage-s of tiles may fit into the buffer of the transfer protocol.
• capturing period may locally be optimized for tiles with high entropy (-> Capturing Strategy).

Further traffic minimization

• Consider differences of subsequent Desktop contents, only
• Compression of tiles by PNG or a PCX-like method

For sharing over internet, Peer-to-Peer connection may be established by

• a public proxy server
• UDP hole punching with, for connection establishing, a necessarily public mediator server

In any case the protocol needs to be secured and delivery asserted.