[NTLUG:Discuss] Apache 1.3.27 w/ RedHat 7.3 driving....re-visited

[Douglas King]

I have begun having a problem with my Apache server. The problem is as
follows:

I am not able to run / execute ANY Perl or cgi scripts...at all. Right now, 
I cannot even ADD another site. This server IS RUNNING approximately 1,100 
sites...MOST of them are virtual (second level domain hosting) 
http://www.xyz.abc.com type hosting....

In the error log for Apache, I get the following error message:

[Tue Nov 26 21:47:33 2002] [warn] (24)Too many open files: unable to open a
file descriptor above 15, you may need to increase the number of descriptors
[Tue Nov 26 21:47:34 2002] [notice] Apache/1.3.23 (Unix) (Red-Hat/Linux)
mod_ssl/2.8.7 OpenSSL/0.9.6b DAV/1.0.3 PHP/4.1.2 mod_perl/1.26 configured
-- resuming normal operations
[Tue Nov 26 21:47:34 2002] [notice] suEXEC mechanism enabled (wrapper:
/usr/sbin/suexec)
[Tue Nov 26 21:47:34 2002] [notice] Accept mutex: sysvsem (Default: sysvsem)
[Tue Nov 26 21:47:52 2002] [error] [client 66.14.30.174] (24)Too many open
files: couldn't spawn child process:
/var/www/cgi-bin/domainname/mycgi_list.cgi

AND more recently:

[Wed Dec 18 19:59:48 2002] [crit] [client 64.12.96.73] (24)Too many open 
files: /.htaccess pcfg_openfile: unable to check htaccess file, ensure it 
is readable
[Wed Dec 18 20:00:02 2002] [crit] [client 64.12.96.73] (24)Too many open 
files: /.htaccess pcfg_openfile: unable to check htaccess file, ensure it 
is readable
[Wed Dec 18 20:00:19 2002] [crit] [client 66.69.18.234] (24)Too many open 
files: /.htaccess pcfg_openfile: unable to check htaccess file, ensure it 
is readable
[Wed Dec 18 20:00:23 2002] [crit] [client 64.12.96.7] (24)Too many open 
files: /.htaccess pcfg_openfile: unable to check htaccess file, ensure it 
is readable
[Wed Dec 18 20:00:33 2002] [crit] [client 64.12.96.8] (24)Too many open 
files: /.htaccess pcfg_openfile: unable to check htaccess file, ensure it 
is readable
[Wed Dec 18 20:00:34 2002] [crit] [client 64.12.97.12] (24)Too many open 
files: /.htaccess pcfg_openfile: unable to check htaccess file, ensure it 
is readable
[Wed Dec 18 20:01:19 2002] [crit] [client 209.237.238.160] (24)Too many 
open files: /.htaccess pcfg_openfile: unable to check htaccess file, ensure 
it is readable
[Wed Dec 18 20:02:24 2002] [crit] [client 150.137.68.68] (24)Too many open 
files: /.htaccess pcfg_openfile: unable to check htaccess file, ensure it 
is readable
[Wed Dec 18 20:03:25 2002] [notice] caught SIGTERM, shutting down
[Wed Dec 18 20:03:29 2002] [warn] (24)Too many open files: unable to open a 
file descriptor above 15, you may need to increase the number of descriptors
[Wed Dec 18 20:03:30 2002] [notice] Apache/1.3.27 (Unix) (Red-Hat/Linux) 
mod_ssl/2.8.7 OpenSSL/0.9.6b DAV/1.0.3 PHP/4.1.2 mod_perl/1.26 configured 
-- resuming normal operations
[Wed Dec 18 20:03:30 2002] [notice] suEXEC mechanism enabled (wrapper: 
/usr/sbin/suexec)
[Wed Dec 18 20:03:30 2002] [notice] Accept mutex: sysvsem (Default: sysvsem)
[Wed Dec 18 20:03:32 2002] [error] (24)Too many open files: accept: (client 
socket)
[Wed Dec 18 20:03:32 2002] [error] (24)Too many open files: accept: (client 
socket)
[Wed Dec 18 20:03:34 2002] [error] (24)Too many open files: accept: (client 
socket)
[Wed Dec 18 20:03:40 2002] [error] (24)Too many open files: accept: (client 
socket)
[Wed Dec 18 20:03:46 2002] [error] (24)Too many open files: accept: (client 
socket)
[Wed Dec 18 20:03:46 2002] [error] (24)Too many open files: accept: (client 
socket)
[Wed Dec 18 20:03:46 2002] [error] (24)Too many open files: accept: (client 
socket)
[Wed Dec 18 20:03:51 2002] [error] (24)Too many open files: accept: (client 
socket)
[Wed Dec 18 20:04:28 2002] [notice] caught SIGTERM, shutting down
[Wed Dec 18 20:04:31 2002] [warn] (24)Too many open files: unable to open a 
file descriptor above 15, you may need to increase the number of descriptors
[Wed Dec 18 20:04:32 2002] [notice] Apache/1.3.27 (Unix) (Red-Hat/Linux) 
mod_ssl/2.8.7 OpenSSL/0.9.6b DAV/1.0.3 PHP/4.1.2 mod_perl/1.26 configured 
-- resuming normal operations
[Wed Dec 18 20:04:32 2002] [notice] suEXEC mechanism enabled (wrapper: 
/usr/sbin/suexec)
[Wed Dec 18 20:04:32 2002] [notice] Accept mutex: sysvsem (Default: sysvsem)


Here is the info on the server:

It's an Athalon 650 mhz processor

1 gig ram

It's a Red Hat 7.3 machine running Apache 1.3.27 for the web server.

~~~~~~~~~
When you do a ulimit -a you get:
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
file size (blocks, -f) unlimited
max locked memory (kbytes, -l) unlimited
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 7168
virtual memory (kbytes, -v) unlimited


The cgi scripts used to run...but have been stopped now for almost a month. 
The kernel, 2.4.19
was recently re-configured to basically have unlimited resources 
available...and the file limit or process was set to something like 1024.

Has anybody run across this problem? If so....what is my solution?