When you are confronted with a memory issue, there are certain tools you can use to analyze the memory usage.
One such tool, when using an IBM JVM, is the IBM Heap Analyzer.
This tool is described on this page:
On this page, you can find a link to the latest ha*.jar, as well as a webcast (audio and slides separately) about the original tool.
Here is an example of the steps one can take to analyze a problem. The problem in this example is DERBY-6096. These are the steps:
- Download the heap analyzer and watch the webcast that covers the heap analyzer features.
Obtain an IBM jvm with a version > 1.4.2, we suggest 1.6 or 1.7.
- Obtain derby 10.10 and add the derbyrun.jar to your CLASSPATH environment variable.
download the reproduction for DERBY-6096 (https://issues.apache.org/jira/browse/DERBY-6096) and compile and run it:
- java -Xmx64M D6096
This program will hit an OutOfMemory error which will automatically generate a heap dump. The heap dump will be in a file with extension .phd. The name of this file will depend on the OS.
- start the heap analyzer like so:
java -Xmx1200m -jar <path to ha jar>/ha450.jar
Choose File -> Open and select the heap dump file that you want to inspect.
- Note that the summary shows the memory available to the jvm, the jvm version, and other interesting details.
- Select the tab 'leak suspects'
- Note: With Derby the 'page cache' is intentionally allocated ahead of time. This is almost always identified as a leak suspect and is a red herring, so the first leak suspect can be ignored:
33,729,032 bytes (53.69 %) of Java heap is used by 1,000 instances of org/apache/derby/impl/services/cache/ClockPolicy$Holder
- Look at other leak suspects:
27,039,776 bytes (43.04 %) of Java heap is used by 842 instances of java/util/HashMap$Entry
- Right click on this entry, and select 'Find object in tree view' from the menu.
- You can see these are coming from 27,039,888 (43.04%)  4 org/apache/derby/impl/store/access/BackingStoreHashTableFromScan 0x36c94b8,
- This then gives us a suggested place to start looking at the source code.
In the case of DERBY-6096 the problem was that BLOB size was estimated at 0, and thus the memory usage needed by BLOBs was estimated as small, leading Derby to try to hold large objects in memory when trying to do a hash join