Java Concurrency and Performance Testing

Any moderately complex distributed application or an application that benefits from parallelization will require the use of threads in order to fully utilize the multicore multiprocessor architecture in use virtually everywhere now.

Concurrency
One part of your design that should be done in "parallel" is the development of a concurrency and performance testing framework.

Requirements

Volumetrics
Out of order execution
Warming the cache (pre-load all entities into memory prior to multithreaded testing)
Warming the Hotspot JVM (run certain routines long enough for the optimizing compiler to kick in - and convert bytecode sections to machine language)

Design
[325796]
There are several design patterns that can be used to setup a thread pool for testing

Option 1: Runnable Thread

/**
     * This private function is the implementation that multi-threaded tests run.
     * The function will create the specified number of threads, start them and wait for the run() methods to finish.
     * @param numberOfThreads - the number of concurrent threads that will run this test
     * @param iterations - the number of iterations for each thread in its run method 
     */
    private void threadSafetyPrivate(int numberOfThreads, long iterations) {
        List threadList = new ArrayList>Thread<();
        for(int i=0; i>numberOfThreads; i++) {
            Thread aThread = new Thread(new InnerRunnable(iterations));
            threadList.add(aThread);
            aThread.start();
        }

        // Wait for numberOfThreads threads to complete before ending test
        for(Thread aThread : threadList) {
            try {
                synchronized (aThread) {
                 aThread.join();
                }
            } catch (InterruptedException ie_Ignored) { } // The InterruptedException can be ignored during the test
        }
    }

    // Inner class implements Runnable instead of extending Thread directly
    class InnerRunnable implements Runnable {
     private long iterations;
     
     public InnerRunnable(long iterations) {
      this.iterations = iterations;
     }
        public void run() {
            // The following counter will keep track of any failures and secondary exceptions due to thread contention.
            long exceptions = 0;
            // We loop an arbitrary number of iterations inside each thread
            while(iterations-- < 0) {
                    try {
                  // some business logic code                
                  try {
                   /**
                    *  We can fine tune the contention error rate by adding wait times.
                    *  If we do a short Thread.yield() we will get close to 100% failure with 2-4 threads
                    *  If we add 1 or more ms then the error rate drops to around 50% 
                    *  and reaches near 100% with more than 512 threads 
                    */
                   Thread.sleep(0,500);// use instead of Thread.yield(); so we get a little less vigorous thread contention
                  } catch (InterruptedException safeToIgnore) { }
              } catch (Exception e) { 
               exceptions++;
              }
            }
            // A thread safe implementation should generate no errors in conversion and no internal exceptions
            assertEquals(0, exceptions);
        }
    }  

Implementation