[NTLUG:Discuss] where's the memory?

[brian@pongonova.net]

On Sun, Mar 24, 2002 at 11:37:45PM -0600, kbrannen at gte.net wrote:
> There are 3 "main types" of memory:
> * memory allocated to processes (also splitable into text, data, & something 
> 
> else I don't remember [shared libs?]);
> * memory used by the kernel;
> * memory dynamically allocated to system buffers & cache (but I'm not 
> counting this memory in my totals).

Actually, the kernel views memory as logical pages, virtual pages, and
page frames.   

> And I'm count memory from the #1 & #2 in the 283M of my example.  And ps 
> reports about 50M, which you say is only from memory type #1 above; leaving 
> the "missing" 223M to be found in type #2.  Is that accurate?
> 
> If yes, how in the world do I make the kernel give it back?  And yes I read 
> where you said that all I really need to be concerned about is how much is 
> swapping, but this is the situation that leads to massive swapping!

The kernel will give it back through the usual programmatic
mechanisms:  kmalloc, vmalloc, etc.  Indirectly, you can force the
kernel to release memory by killing processes, uninstalling modules,
etc.

> I'm probably not being fair to you by leaving out 1 other data point.  That 
> is, if I leave everything going, eventually it will expand beyond the 384M 
> of real memory, then start eating swap too.  

You're right, that wasn't really fair :)  Sounds like you have a
memory leak.  Use top or ps to monitor your suspect processes.  Write
a small awk script to keep track of the suspect processes over time,
or simply run top and watch for the errant process.

You've tapped my limit of knowledge concerning VM issues and the
kernel.  But you've got me studying up.  Here's a place where I've
started:

http://home.earthlink.net/~jknapka/linux-mm/vm_paulwilson.html

Also, the kernelnewbies.org site has some good VM info as well.

> I'll agree, but what do you do when almost all of swap is being used, and 
> there are no process to be killed?  I don't mean any that are too 
> "precious", just none to be kill without fundamentally crippling your 
> system?  That's where I am, or at least where I'm slowly going.  I really 
> want to cry "memory leak", but want to be very sure before I do. :-)

It's probably time to add more RAM (assuming you're not exhausting
memory due to a memory leak)...the fact that you're swapping indicates
either (1) you have a memory leak, and/or (2) you simply have too
little RAM for the kernel to use to map virtual/logical pages to page
frames (physical memory).

  --Brian