Java SoftReference, panicing GC and GC behavior

Question

I want to write a cache using SoftReferences using as much memory as possible, as long as it doesn't get too inefficient.

Trying to estimate the used size by calculating object sizes or by getting some used memory approximation of the JVM are dead ends.

The javadoc even states that SoftReferences are good for memory-aware caches, but there is no hard rule on how a JVM implementation shall handle SoftReferences. I'm only talking about the Oracle implementation of the JVM (Version 6.22 and above and Version 7).

Now my questions (please feel free to answer partial, grouped or in any way you please):

1. Does the JVM take the last access of the object into account and only remove the old ones? Javadoc states: Virtual machine implementations are, however, encouraged to bias against clearing recently-created or recently-used soft references.
2. What happens when memory gets tight? The JVM panics and just eats all objects?
3. Is there a parameter for telling the JVM to only eat as much to survive (no OOMEs) and live healthy (not having the CPU only run the GC)

Answer 1

I don't think there is an order. (I'm not sure though about the order of events)

But what happens with soft references is that it is always guaranteed that they will be released before there is an out of memory exception. Unless you have a hard reference pointing to them.

But you should be aware that you might try to access them and they are gone. My guess is that the garbage collector will just eat the first soft reference that fits the amount needed for the operation.

Answer 2

Although SoftReferences are a cool feature, I personally don't dare using them in large projects where I don't know the memory requirements of every other component. Will a memory-hogging SoftReference cache make other parts perform badly?

I'd instead of using SoftReferences I'd consider using EHCache. It let's you limit the size of particular caches in terms of number of entries, or even better, the bytes used in memory (this is a new feature in the upcoming version 2.5). Different eviction strategies can be configured, of course, such as LRU. There's lots you can configure with EHCache.

If you're using Spring, then version 3.1 will also provide you with some nice @Cachable method-level annotations; EHCache can be used as a caching implementation there.

Answer 3

What happens when memory gets tight? The JVM panics and just eats all objects?

I know for a fact that with Oracle 1.6 JVM this is not the case. I am aware of a situation where a server that processes concurrent requests uses a response the contains the actual data inside a soft reference. I have observed that when a low memory situation is reported by one thread the other threads' soft references continue to hold on to their contents (the referenced objects).

Is there a parameter for telling the JVM to only eat as much to survive (no OOMEs) and live healthy (not having the CPU only run the GC)

What is enough to survive? You mean that if X amount of memory is required then only reclaim soft-references till X is available? I didn't find any such tuning parameter but as I said JVM does not seem to be reclaiming all soft references when it needs to reclaim one.