While it doesn't look like that tail call optimization will ever be added as a common optimization technique, especially after Sun was bought, wouldn't it be technically possible to let languages running on the VM decide themselves if their compiler emits tailcall instructions in the byte code?
E. g. Java, Groovy can decide not to use the instruction, while more functional languages like Scala or Clojure can emit it and the HotSpot VM will only optimize the ones marked with tailcall?
Yes, this is technically possible -- in fact there an experimental patch by Arnold Schwaighofer for OpenJDK which does just that, although it isn't easy to apply the patch and build, as it isn't kept up to date at present.
An explicit tail call instruction has advantages over transparent optimisation of function calls in tail position by the JVM, as the JVM can verify that what you have specified as a tail call really is see this tail call blog post by John Rose for an explanation of some of the verification which can by done.
I think the future of the JVM is not Java, and I hope that someone with the time and talent pushes tail calls for the JVM forward.