Memory leak

[Behdad Esfahbod]

On Mon, 12 Dec 2005, Lennart Sorensen wrote:

> On Sun, Dec 11, 2005 at 12:56:41PM -0500, billt-lxSQFCZeNF4 at public.gmane.org wrote:

> > Detecting it is really a matter of running the application
> > and noticing that it is using more and more memory as time
> > goes on. There are some compilers that try to detect it but
> > to my knowledge there is no full proof way doing that.

Note that if the so-called size of the process keeps increasing
over time, it does not necessarily mean that the application is
leaking.  That's in fact a known phenamenon about Firefox.  The
problem is that malloc() implementation for not-so-large pieces
of memory (like < 4kb) uses the heap for allocation.  If there's
not enough space available in the heap, it will use the system
call brk(2) to increase the size of the heap of the process.  Now
note that the heap is a linear chunk in the address space of the
process, it grows at the end, and shrink at the end.  So at
free() time, if a minimum of memory at the *end* of the heap is
free, it will be returned to the system, reducing the size of the
process.

Now consider the following situation: you open a bunch of very
very long pages in Firefox.  All the laying out of text,
javascripts, etc, eats up a good 100MB of your memory, so the
firefox process grows to something like 120MB.  Now you open a
tab, loading a very small page in it.  The memory allocations for
this small page most probably happen to be after the 100MB
allocations for your long pages.  Now you close all the long
pages, the memory they have allocated using malloc() is free()d
and returned to the libc, but since the memory for the small page
is not free()d yet, you have some allocated memory at the end of
your heap, which has a good 101MB of size now.  So even though
Firefox have free()d 100MB of the 101MB, it will *not* be
returned to the system and the size of the firefox process will
remain around 121MB.

Now this is not the end of the story, there's the issue of
fragmentation too.  Left as an exercise.

There are several attempts to fix these problems.  The recent
g_slice allocated in glib is one of them:

  http://www.mail-archive.com/gtk-devel-list-rDKQcyrBJuzYtjvyW6yDsg@public.gmane.org/msg01723.html

The idea behind this and other similar allocators is that instead
of allocating from the heap, the serve memory allocations of the
same size from a small pool allocated specifically for slices of
that range of size.  These pools on the other hand, are around a
pagesize (4kb) in size and so are allocated using mmap(2), not
brk(2).  mmap()ed allocations can be freed and returned to system
at any time, unlike heap memory.


> I remember using purify at university on solaris a few times.
> It is absolutely amazing for doing such things (and other array
> bounds checks and such).  It appears it is now available for
> linux, and is only about $8000 per license. :)

Valgrind is an amazing Free Software project doing along the same
lines as purify.


--behdad
http://behdad.org/

"Commandment Three says Do Not Kill, Amendment Two says Blood Will Spill"
	-- Dan Bern, "New American Language"
--
The Toronto Linux Users Group.      Meetings: http://tlug.ss.org
TLUG requests: Linux topics, No HTML, wrap text below 80 columns
How to UNSUBSCRIBE: http://tlug.ss.org/subscribe.shtml