• !! This page is under construction !!*

For some time Tomcat has had some means of protection against memory leaks when stopping or redeploying applications. This page tries to list them, and shows the situations where leaks can be detected and fixed.

Diagnose a classloader leak upon request

Starting with tomcat 6.0.25, the manager webapp has a new "Find Leaks" button. When triggered, it displays a list of webapps (their context path) that have been stopped (this includes undeployed and redeployed ones) but whose classloader failed to be GCed.

If a leaking webapp is redeployed several times, it will appear as many times as it actually leaked.

Caution: This diagnosis calls System.gc() which may not be desirable in production environments.

Different types of leaks that Tomcat can detect (or not)

When a webapp execution is stopped (this encompassed redeploy and undeploy), tomcat tries to detect and fix leaks.

Starting with tomcat 6.0.24, messages are logged to indicate the kind of leak that was detected.

ThreadLocal leaks

Classloader leaks because of uncleaned ThreadLocal variables are quite common. Depending on the use cases, they can be detected or not.

Custom ThreadLocal class

Suppose we have the following 3 classes in our webapp :

public class MyCounter {
	private int count = 0;

	public void increment() {
		count++;
	}

	public int getCount() {
		return count;
	}
}

public class MyThreadLocal extends ThreadLocal<MyCounter> {
}

public class LeakingServlet extends HttpServlet {
	private static MyThreadLocal myThreadLocal = new MyThreadLocal();

	protected void doGet(HttpServletRequest request,
			HttpServletResponse response) throws ServletException, IOException {

		MyCounter counter = myThreadLocal.get();
		if (counter == null) {
			counter = new MyCounter();
			myThreadLocal.set(counter);
		}

		response.getWriter().println(
				"The current thread served this servlet " + counter.getCount()
						+ " times");
		counter.increment();
	}
}

If the LeakingServlet is invoked at least once and the Thread that served it is not stopped, then we created a classloader leak !

The leak is caused because we have a custom class for the ThreadLocal instance, and also a custom class for the value bound to the Thread. Actually the important thing is that both classes were loaded by the webapp classloader.

Hopefully tomcat 6.0.24 can detect the leak when the application is stopped: each Thread in the JVM is examined, and the internal structures of the Thread and ThreadLocal classes are introspected to see if either the ThreadLocal instance or the value bound to it were loaded by the WebAppClassLoader of the application being stopped.

In this particular case, the leak is detected, a message is logged and internal structures of the JDK (ThreadLocalMap) are modified to remove the reference to the ThreadLocal instance.

Mar 16, 2010 11:47:24 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
SEVERE: A web application created a ThreadLocal with key of type [test.MyThreadLocal] (value [test.MyThreadLocal@4dbb9a58]) and a value of type [test.MyCounter] (value [test.MyCounter@57922f46]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.

Note: this particular leak was actually already cured by previous versions of tomcat, because static references of classes loaded by the webappclassloader are nullified (see later).

Webapp class instance as ThreadLocal value

Suppose that we have the following class in the common classpath (for instance in a jar in tomcat/lib) :

public class ThreadScopedHolder {
	private final static ThreadLocal<Object> threadLocal = new ThreadLocal<Object>();

	public static void saveInHolder(Object o) {
		threadLocal.set(o);
	}

	public static Object getFromHolder() {
		return threadLocal.get();
	}
}

And those 2 classes in the webapp :

public class MyCounter {
	private int count = 0;

	public void increment() {
		count++;
	}

	public int getCount() {
		return count;
	}
}
public class LeakingServlet extends HttpServlet {

	protected void doGet(HttpServletRequest request,
			HttpServletResponse response) throws ServletException, IOException {

		MyCounter counter = (MyCounter)ThreadScopedHolder.getFromHolder();
		if (counter == null) {
			counter = new MyCounter();
			ThreadScopedHolder.saveInHolder(counter);
		}

		response.getWriter().println(
				"The current thread served this servlet " + counter.getCount()
						+ " times");
		counter.increment();
	}
}

If the servlet is invoked at least once, the webapp classloader would not be GCed when the app is stopped: since the classloader of ThreadScopedHolder is the common classloader, it remains forever which is as expected. But its ThreadLocal instance has a value bound to it (for the non-terminated thread(s) that served the sevlet), which is an instance of a class loaded by the webapp classloader...

Here again, tomcat 6.0.24 will detect and fix the leak by removing this ThreadLocal reference from each Thread.

Mar 17, 2010 10:23:13 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
SEVERE: A web application created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@44676e3f]) and a value of type [test.leak.threadlocal.value.MyCounter] (value [test.leak.threadlocal.value.MyCounter@62770d2e]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.

Webapp class instance indirectly held through a ThreadLocal value

Suppose we have the same ThreadScopedHolder class (in the common classloader) and MyCounter class in the webapp, but with the following servlet :

public class LeakingServlet extends HttpServlet {

	protected void doGet(HttpServletRequest request,
			HttpServletResponse response) throws ServletException, IOException {

		List<MyCounter> counterList = (List<MyCounter>) ThreadScopedHolder
				.getFromHolder();
		MyCounter counter;
		if (counterList == null) {
			counter = new MyCounter();
			ThreadScopedHolder.saveInHolder(Arrays.asList(counter));
		} else {
			counter = counterList.get(0);
		}

		response.getWriter().println(
				"The current thread served this servlet " + counter.getCount()
						+ " times");
		counter.increment();
	}
}

We have more or less the same kind of leak as the previous one, but this time tomcat 6.0.24 does not detect the leak when stopping the application (and does not fix it). The problem is that when it inspects the entries of ThreadLocalMap, it checks whether either the key or the value is an instance of a class loaded by the webapp classloader. Here the key is an instance of ThreadLocal, and the value is an instance of java.util.ArrayList.

The "Find leaks" button in tomcat manager will report the leak when asked :

The following web applications were stopped (reloaded, undeployed), but their
classes from previous runs are still loaded in memory, thus causing a memory
leak (use a profiler to confirm):
/testWeb

But it does not give any clue about what caused the leak, we would need to make a heapdump and analyse it with some tool like Eclipse MAT.

ThreadLocal pseudo-leak

Threads ContextClassLoader

Threads spawned by JRE classes

Threads spawned by classes loaded by the common classloader

Threads spawned by webapps

Child classloaders

static class variables

LogFactory

JavaBean Introspector cache

JDBC driver registration

RMI target

Summary matrix

References

Mark Thomas interview on DZone

Related issues

• 48837 - Memory leaks protection does not cure leaks triggered by JSP pages code