Hmm, that looks at around the right time for a logrotate, which includes a
graceful to apache... I'd guess that there was something making apache very
very unhappy - but it's hard to tell from that!
Secondly, I really ought to have some mechanism to check on Apache (and MySQL) periodically and make sure they're running, either restarting them or alerting me (although I can be difficult to get hold of). What do people recommend for this? Is it as simple as having cron run a script to start the daemon if it's not already on the stored PID, or are there issues I should be wary of?
I'd test the services as apposed to the PIDs that you think they should
have... throw a test page up on apache with some known content and then test
against that every once in a while... I'm also not one for
automatically restarting services - it masks potential issues which then may go unnoticed for a > long time. We use nagios + clickatell (sms
service) to monitor our systems - with a period between about 11.30pm and
6.30am where smses aren't sent (thank god!).
I've battled with exactly this on a virtual server I look after. There are numerous bug reports, fixes and workarounds out there for the taking. The only certain thing is that YMMV.
My fix was probably related to the fact that apache is running in a virtual server, but here it is FWIW:
/tmp was defined as a tmpfs in /etc/fstab. After commenting this out (bearing in mind the performance impact) and rebooting, I've not had any problems (yet?).
Here's where I started my journey: http://www.bytemark.co.uk/page/Live/support/tech/inside/apachereload
HTH
Safe