Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Leak cause

Detected by tomcat

Fixed by tomcat

Possible enhancements

Custom ThreadLocal class

>=6.0.24>=

6.0.24-6.0.26 but is unsafe. Made optional from 6.0.27

Detect child classloaders, improve thread safety BZ 49159

Webapp class instance as ThreadLocal value

>=6.0.24>=

6.0.24-6.0.26 but is unsafe. Made optional from 6.0.27

Detect child classloaders, improve thread safety BZ 49159

Webapp class instance indirectly held through a ThreadLocal value

no

no

Renew threads in the worker pool

ThreadLocal pseudo-leak

>=6.0.24>=

6.0.24

improve thread safety

ContextClassLoader / Threads spawned by webapps

>=-6.0.2426 but is unsafe. Made optional from 6.0.27

BZ 49159

ContextClassLoader / Threads spawned by webapps

>=6.0.24

In 6.0.24-6.0.26 TimerThread are stopped but it may lead to problems. Optional from 6.0.27 with the clearReferencesStopTimerThreads flag. TimerThread are stopped. Other threads may be stopped if clearReferencesStopThreads is set to true on the Context with the clearReferencesStopThreads flag, but it's unsafe.

Detect child classloaders, don't stop TimerThread by defaultFix the application to stop the thread when the application is stopped

ContextClassLoader / Threads spawned by classes loaded by the common classloader

>=6.0.24

In 6.0.24-6.0.26 TimerThread are stopped but it may lead to problems. Optional from 6.0.27 with the clearReferencesStopTimerThreads flag. Other threads may be stopped if clearReferencesStopThreads is set to true on the Context with the clearReferencesStopThreads flag, but it's unsafe.

Detect child classloaders, don't stop TimerThread by defaultfix the offending code (set the correct CCL when spawning the thread)

ContextClassLoader / Threads spawned by JRE classes

no

>=6.0.24 pre-spawns some known offenders

 

static class variables

no

> 6.0.?

 

LogFactory

 

> 6.0.?

 

JDBC driver registration

> 6.0.?

> 6.0.?

 

RMI Target

 

> 6.0.?

 

...

In this particular case, the leak is detected , and a message is logged and . Tomcat 6.0.24 to 6.0.26 modify internal structures of the JDK (ThreadLocalMap) are modified to remove the reference to the ThreadLocal instancereference to the ThreadLocal instance, but this is unsafe (see #48895) so that it became optional and disabled by default from 6.0.27.

No Format
Mar 16, 2010 11:47:24 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
SEVERE: A web application created a ThreadLocal with key of type [test.MyThreadLocal] (value [test.MyThreadLocal@4dbb9a58]) and a value of type [test.MyCounter] (value [test.MyCounter@57922f46]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.

...

If the servlet is invoked at least once, the webapp classloader would not be GCed when the app is stopped: since the classloader of ThreadScopedHolder is the common classloader, it remains forever which is as expected. But its ThreadLocal instance has a value bound to it (for the non-terminated thread(s) that served the sevlet), which is an instance of a class loaded by the webapp classloader...

Here again, tomcat >=6.0.24 will detect and fix the leak by removing this ThreadLocal reference from each Thread.:

No Format
Mar 17, 2010 10:23:13 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
SEVERE: A web application created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@44676e3f]) and a value of type [test.leak.threadlocal.value.MyCounter] (value [test.leak.threadlocal.value.MyCounter@62770d2e]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.

...

If many threads were used to serve our leaking webapp, but after we stop it only a couple of threads are enough to serve other webapps, one could have some threads that are no longer used, waiting for some work. Since those threads are blocked, they have no interaction with their ThreadLocalMap (i.e. there's no ThreadLocal value bound to them or removed), so that there's no opportunity to expungeStaleEntries().

Tomcat 6.0.24-6.0.26 "speeds up" the removal of stale entries (and thus fixes the pseudo-leak), by calling expungeStaleEntries() for each thread that has some stale entries. Since it's not thread-safe, it has been made optional and disabled by default from 6.0.27.

Threads ContextClassLoader

...

So the leak is fixed, but unfortunately there's a side effect : it broke webapp B eviction timer.Note: as of 6.0.24, by default tomcat stops threads of class java.util.TimerThread whose context classloader is the WebAppClassLoader of the app being stopped. It does not stop other threads, it only warns about them. It can try to stop them if the clearReferencesStopThreads option of the standard context is set to truewebapp B eviction timer. That's why stopping TimerThread has been made optional from 6.0.27.

Anchor
cclThreadSpawnedByJRE
cclThreadSpawnedByJRE

...

When an app is stopped, Tomcat 6.0.24 detects and later detect leaks caused by ThreadLocal}}s and Threads context classloader only by checking for the current {{WebAppClassLoader. If a child classloader is involved, the leak is not detected. That should be improved in a future release.

...

  • Anchor
    48837
    48837
    48837 - Memory leaks protection does not cure leaks triggered by JSP pages code
  • Anchor
    48895
    48895
    48895 - WebAppClassLoader.clearThreadLocalMap() concurrency issues
  • Anchor
    48971
    48971
    48971 - memory leak protection : stopping TimerThreads should be optional and disabled by default
  • Anchor
    49159
    49159
    49159 - Improve ThreadLocal memory leak clean-up
  • Sun bug 4957990 - In some cases the Server JVM fails to collect classloaders. According to this page it should have been fixed with java 6u16 but actually it was not. It seems to be fixed with 6u21 (documented here and verified by the author of this wiki page).