How can I create a memory leak in Java?

ID : 128

viewed : 303

Tags : javamemorymemory-leaksjava





Top 5 Answer for How can I create a memory leak in Java?

vote vote

95

Here's a good way to create a true memory leak (objects inaccessible by running code but still stored in memory) in pure Java:

  1. The application creates a long-running thread (or use a thread pool to leak even faster).
  2. The thread loads a class via an (optionally custom) ClassLoader.
  3. The class allocates a large chunk of memory (e.g. new byte[1000000]), stores a strong reference to it in a static field, and then stores a reference to itself in a ThreadLocal. Allocating the extra memory is optional (leaking the class instance is enough), but it will make the leak work that much faster.
  4. The application clears all references to the custom class or the ClassLoader it was loaded from.
  5. Repeat.

Due to the way ThreadLocal is implemented in Oracle's JDK, this creates a memory leak:

  • Each Thread has a private field threadLocals, which actually stores the thread-local values.
  • Each key in this map is a weak reference to a ThreadLocal object, so after that ThreadLocal object is garbage-collected, its entry is removed from the map.
  • But each value is a strong reference, so when a value (directly or indirectly) points to the ThreadLocal object that is its key, that object will neither be garbage-collected nor removed from the map as long as the thread lives.

In this example, the chain of strong references looks like this:

Thread object → threadLocals map → instance of example class → example class → static ThreadLocal field → ThreadLocal object.

(The ClassLoader doesn't really play a role in creating the leak, it just makes the leak worse because of this additional reference chain: example class → ClassLoader → all the classes it has loaded. It was even worse in many JVM implementations, especially prior to Java 7, because classes and ClassLoaders were allocated straight into permgen and were never garbage-collected at all.)

A variation on this pattern is why application containers (like Tomcat) can leak memory like a sieve if you frequently redeploy applications which happen to use ThreadLocals that in some way point back to themselves. This can happen for a number of subtle reasons and is often hard to debug and/or fix.

Update: Since lots of people keep asking for it, here's some example code that shows this behavior in action.

vote vote

82

Static field holding an object reference [especially a final field]

class MemorableClass {     static final ArrayList list = new ArrayList(100); } 

Calling String.intern() on a lengthy string

String str = readString(); // read lengthy string any source db,textbox/jsp etc.. // This will place the string in memory pool from which you can't remove str.intern(); 

(Unclosed) open streams (file , network, etc.)

try {     BufferedReader br = new BufferedReader(new FileReader(inputFile));     ...     ... } catch (Exception e) {     e.printStacktrace(); } 

Unclosed connections

try {     Connection conn = ConnectionFactory.getConnection();     ...     ... } catch (Exception e) {     e.printStacktrace(); } 

Areas that are unreachable from JVM's garbage collector, such as memory allocated through native methods.

In web applications, some objects are stored in application scope until the application is explicitly stopped or removed.

getServletContext().setAttribute("SOME_MAP", map); 

Incorrect or inappropriate JVM options, such as the noclassgc option on IBM JDK that prevents unused class garbage collection

See IBM JDK settings.

vote vote

72

A simple thing to do is to use a HashSet with an incorrect (or non-existent) hashCode() or equals(), and then keep adding "duplicates". Instead of ignoring duplicates as it should, the set will only ever grow and you won't be able to remove them.

If you want these bad keys/elements to hang around you can use a static field like

class BadKey {    // no hashCode or equals();    public final String key;    public BadKey(String key) { this.key = key; } }  Map map = System.getProperties(); map.put(new BadKey("key"), "value"); // Memory leak even if your threads die. 
vote vote

64

Below there will be a non-obvious case where Java leaks, besides the standard case of forgotten listeners, static references, bogus/modifiable keys in hashmaps, or just threads stuck without any chance to end their life-cycle.

  • File.deleteOnExit() - always leaks the string, if the string is a substring, the leak is even worse (the underlying char[] is also leaked) - in Java 7 substring also copies the char[], so the later doesn't apply; @Daniel, no needs for votes, though.

I'll concentrate on threads to show the danger of unmanaged threads mostly, don't wish to even touch swing.

  • Runtime.addShutdownHook and not remove... and then even with removeShutdownHook due to a bug in ThreadGroup class regarding unstarted threads it may not get collected, effectively leak the ThreadGroup. JGroup has the leak in GossipRouter.

  • Creating, but not starting, a Thread goes into the same category as above.

  • Creating a thread inherits the ContextClassLoader and AccessControlContext, plus the ThreadGroup and any InheritedThreadLocal, all those references are potential leaks, along with the entire classes loaded by the classloader and all static references, and ja-ja. The effect is especially visible with the entire j.u.c.Executor framework that features a super simple ThreadFactory interface, yet most developers have no clue of the lurking danger. Also a lot of libraries do start threads upon request (way too many industry popular libraries).

  • ThreadLocal caches; those are evil in many cases. I am sure everyone has seen quite a bit of simple caches based on ThreadLocal, well the bad news: if the thread keeps going more than expected the life the context ClassLoader, it is a pure nice little leak. Do not use ThreadLocal caches unless really needed.

  • Calling ThreadGroup.destroy() when the ThreadGroup has no threads itself, but it still keeps child ThreadGroups. A bad leak that will prevent the ThreadGroup to remove from its parent, but all the children become un-enumerateable.

  • Using WeakHashMap and the value (in)directly references the key. This is a hard one to find without a heap dump. That applies to all extended Weak/SoftReference that might keep a hard reference back to the guarded object.

  • Using java.net.URL with the HTTP(S) protocol and loading the resource from(!). This one is special, the KeepAliveCache creates a new thread in the system ThreadGroup which leaks the current thread's context classloader. The thread is created upon the first request when no alive thread exists, so either you may get lucky or just leak. The leak is already fixed in Java 7 and the code that creates thread properly removes the context classloader. There are few more cases (like ImageFetcher, also fixed) of creating similar threads.

  • Using InflaterInputStream passing new java.util.zip.Inflater() in the constructor (PNGImageDecoder for instance) and not calling end() of the inflater. Well, if you pass in the constructor with just new, no chance... And yes, calling close() on the stream does not close the inflater if it's manually passed as constructor parameter. This is not a true leak since it'd be released by the finalizer... when it deems it necessary. Till that moment it eats native memory so badly it can cause Linux oom_killer to kill the process with impunity. The main issue is that finalization in Java is very unreliable and G1 made it worse till 7.0.2. Moral of the story: release native resources as soon as you can; the finalizer is just too poor.

  • The same case with java.util.zip.Deflater. This one is far worse since Deflater is memory hungry in Java, i.e. always uses 15 bits (max) and 8 memory levels (9 is max) allocating several hundreds KB of native memory. Fortunately, Deflater is not widely used and to my knowledge JDK contains no misuses. Always call end() if you manually create a Deflater or Inflater. The best part of the last two: you can't find them via normal profiling tools available.

(I can add some more time wasters I have encountered upon request.)

Good luck and stay safe; leaks are evil!

vote vote

57

Most examples here are "too complex". They are edge cases. With these examples, the programmer made a mistake (like don't redefining equals/hashcode), or has been bitten by a corner case of the JVM/JAVA (load of class with static...). I think that's not the type of example an interviewer want or even the most common case.

But there are really simpler cases for memory leaks. The garbage collector only frees what is no longer referenced. We as Java developers don't care about memory. We allocate it when needed and let it be freed automatically. Fine.

But any long-lived application tend to have shared state. It can be anything, statics, singletons... Often non-trivial applications tend to make complex objects graphs. Just forgetting to set a reference to null or more often forgetting to remove one object from a collection is enough to make a memory leak.

Of course all sort of listeners (like UI listeners), caches, or any long-lived shared state tend to produce memory leak if not properly handled. What shall be understood is that this is not a Java corner case, or a problem with the garbage collector. It is a design problem. We design that we add a listener to a long-lived object, but we don't remove the listener when no longer needed. We cache objects, but we have no strategy to remove them from the cache.

We maybe have a complex graph that store the previous state that is needed by a computation. But the previous state is itself linked to the state before and so on.

Like we have to close SQL connections or files. We need to set proper references to null and remove elements from the collection. We shall have proper caching strategies (maximum memory size, number of elements, or timers). All objects that allow a listener to be notified must provide both a addListener and removeListener method. And when these notifiers are no longer used, they must clear their listener list.

A memory leak is indeed truly possible and is perfectly predictable. No need for special language features or corner cases. Memory leaks are either an indicator that something is maybe missing or even of design problems.

Top 3 video Explaining How can I create a memory leak in Java?







Related QUESTION?