Ways to improve performance consistency

I’m not an expert in the area of processor caches but I suspect your issue is essentially a cache issue or some other memory layout problem. Repeated allocation of the buffers and counters without cleaning up the old objects may be causing you to periodically get a very bad cache layout, which may lead to your inconsistent performance.

Using your code and making a few mods I have been able to make the performance consistent (my test machine is Intel Core2 Quad CPU Q6600 2.4GHz w/ Win7x64 – so not quite the same but hopefully close enough to have relevant results). I’ve done this in two different ways both of which have roughly the same effect.

First, move creation of the buffers and counters outside of the doTest method so that they are created only once and then reused for each pass of the test. Now you get the one allocation, it sits nicely in the cache and performance is consistent.

Another way to get the same reuse but with “different” buffers/counters was to insert a gc after the performTiming loop:

for ( int i = 0; i < 3; i++ )
    performTiming ( writeBuffer, readBuffer, readCount, writeCount );
System.out.println ();
System.gc ();

Here the result is more or less the same – the gc lets the buffers/counters be reclaimed and the next allocation ends up reusing the same memory (at least on my test system) and you end up in cache with consistent performance (I also added printing of the actual addresses to verify reuse of the same locations). My guess is that without the clean up leading to reuse you eventually end up with a buffer allocated that doesn’t fit into the cache and your performance suffers as it is swapped in. I suspect that you could do some strange things with order of allocation (like you can make the performance worse on my machine by moving the counter allocation in front of the buffers) or creating some dead space around each run to “purge” the cache if you didn’t want to eliminate the buffers from a prior loop.

Finally, as I said, processor cache and the fun of memory layouts aren’t my area of expertise so if the explanations are misleading or wrong – sorry about that.