Replace your singleton cache with a distributed cache.
One such cache could be JBoss Infinispan but I’m sure that other distributed cache and grid technologies exist, including commercial ones which are probably more mature at this point.
For singleton objects in general, I’m not sure. I think I’d try to not have singletons in the first place.