Quick 'n' dirty warning email if disk usage low
If I want to run a script on my externally hosted server to mark me if my disk space is getting low, what's the simplest way to do it? Say I want a report if "df" reports more than 90% used, I want the script to run regularly but not send an email every time it runs if the problem hasn't worsend, and want to include the output from 'du -h' in the email? Then, once I have that sorted, what's the "best" way to do this kind of thing? Oh, and what's the best way to work out *why* I only have 1% left on my server? -- Mark Rogers More Solutions Ltd :: 0845 45 89 555
On Thu, Jan 26, 2006 at 02:36:17PM +0000, Mark Rogers wrote:
If I want to run a script on my externally hosted server to mark me if my disk space is getting low, what's the simplest way to do it?
Hmm, something like... --- Begin Script #!/bin/bash filesystem="/" warningpercentage=90 persontomail=me@someplace.com subjectofmail='Warning: You are running low on diskspace!' percentageused=$(df -h $filesystem | tail -n 1 | awk '{print $5}') if [[ $percentageused -gt $warningpercentage ]]; then ( echo $nicemessage echo df -h $filesystem ) | mail -s $subjectofmail $persontomail fi --- End Script
Say I want a report if "df" reports more than 90% used, I want the script to run regularly but not send an email every time it runs if the problem hasn't worsend, and want to include the output from 'du -h' in the email?
Then, once I have that sorted, what's the "best" way to do this kind of thing?
cron it half hourly?
Oh, and what's the best way to work out *why* I only have 1% left on my server?
I tend to sit and use du -sh by hand to narrow it down :) Cheers, -- Brett Parker
On 26-Jan-06 Brett Parker wrote:
On Thu, Jan 26, 2006 at 02:36:17PM +0000, Mark Rogers wrote:
If I want to run a script on my externally hosted server to mark me if my disk space is getting low, what's the simplest way to do it?
Hmm, something like...
--- Begin Script #!/bin/bash
filesystem="/" warningpercentage=90 persontomail=me@someplace.com subjectofmail='Warning: You are running low on diskspace!'
percentageused=$(df -h $filesystem | tail -n 1 | awk '{print $5}')
if [[ $percentageused -gt $warningpercentage ]]; then ( echo $nicemessage echo df -h $filesystem ) | mail -s $subjectofmail $persontomail fi
That's surely on the right lines, but since 'df' outputs (e.g.) "91%" and not "91" as the percentage used ("Capacity"), the test will fail (and I don't think "[[" and "]]" will work either). However, the following should be OK: percentageused=$(df -h $filesystem | tail -n 1 | sed 's/%//g' | awk '{print $5}') if [ $percentageused -gt $warningpercentage ]; then ... Best wishes, Ted. -------------------------------------------------------------------- E-Mail: (Ted Harding) <Ted.Harding@nessie.mcc.ac.uk> Fax-to-email: +44 (0)870 094 0861 Date: 26-Jan-06 Time: 16:10:15 ------------------------------ XFMail ------------------------------
On Thu, Jan 26, 2006 at 04:10:18PM -0000, Ted Harding wrote:
On 26-Jan-06 Brett Parker wrote:
On Thu, Jan 26, 2006 at 02:36:17PM +0000, Mark Rogers wrote:
If I want to run a script on my externally hosted server to mark me if my disk space is getting low, what's the simplest way to do it?
Hmm, something like...
--- Begin Script #!/bin/bash
filesystem="/" warningpercentage=90 persontomail=me@someplace.com subjectofmail='Warning: You are running low on diskspace!'
percentageused=$(df -h $filesystem | tail -n 1 | awk '{print $5}')
if [[ $percentageused -gt $warningpercentage ]]; then ( echo $nicemessage echo df -h $filesystem ) | mail -s $subjectofmail $persontomail fi
That's surely on the right lines, but since 'df' outputs (e.g.) "91%" and not "91" as the percentage used ("Capacity"), the test will fail (and I don't think "[[" and "]]" will work either).
[[ and ]] definately do work, I tested that bit and everything! (it's a bashism, IIRC, though, hence starting with #!/bin/bash)
However, the following should be OK:
percentageused=$(df -h $filesystem | tail -n 1 | sed 's/%//g' | awk '{print $5}')
Oh soddit, lets go one better :) percentageused=$(df -h / | sed '1d; s#^.* \([0-9]*\)%.*$#\1#;')
if [ $percentageused -gt $warningpercentage ]; then ...
Hmm, either works. Test it sometime :) -- Brett Parker
Brett Parker wrote:
Hmm, something like...
OK, I'm currently running with: #!/bin/bash warningpercentage=90 persontomail=me@someplace.com subjectofmail='Warning: You are running low on diskspace!' percentageused=$(df -h | sed '1d; s#^.* \([0-9]*\)%.*$#\1#;') if [[ $percentageused -gt $warningpercentage ]]; then ( echo $nicemessage echo df -h ) | mail -s "$subjectofmail" $persontomail fi The only significant issue I ran in to getting this far was missing quotes around $subjectofmail in mail command; I initially sent emails to you@, are@, running@, etc. The bit missing is the fact I want to run it regularly from cron, but not get an email every 30 mins (or whatever) once the disk space runs low. I think it should ideally send a new one every time the %age drops from the last email sent (so if the threshold is 90% I'll get no more than 10 emails before the server fills). But obviously it needs to reset itself when the level drops back down. So, my quick hack to get that is: #!/bin/bash warningpercentage=90 persontomail=me@someplace.com subjectofmail='Warning: You are running low on diskspace!' oldusage=0 if [ -f ~/.diskspacewarning ]; then oldusage=`cat ~/.diskspacewarning` fi percentageused=$(df -h | sed '1d; s#^.* \([0-9]*\)%.*$#\1#;') if [ $percentageused -gt $oldusage ]; then if [[ $percentageused -gt $warningpercentage ]]; then ( echo $nicemessage echo df -h ) | mail -s "$subjectofmail" $persontomail fi fi echo $percentageused > ~/.diskspacewarning It works, but I'm sure it could be done better? [This is a learning exercise for me!] Thanks (all) for the help so far. -- Mark Rogers More Solutions Ltd :: 0845 45 89 555
On Thu, Jan 26, 2006 at 05:13:37PM +0000, Mark Rogers wrote:
Brett Parker wrote:
Hmm, something like...
OK, I'm currently running with:
#!/bin/bash
warningpercentage=90 persontomail=me@someplace.com subjectofmail='Warning: You are running low on diskspace!'
percentageused=$(df -h | sed '1d; s#^.* \([0-9]*\)%.*$#\1#;')
if [[ $percentageused -gt $warningpercentage ]]; then ( echo $nicemessage echo df -h ) | mail -s "$subjectofmail" $persontomail fi
The only significant issue I ran in to getting this far was missing quotes around $subjectofmail in mail command; I initially sent emails to you@, are@, running@, etc.
The bit missing is the fact I want to run it regularly from cron, but not get an email every 30 mins (or whatever) once the disk space runs low. I think it should ideally send a new one every time the %age drops from the last email sent (so if the threshold is 90% I'll get no more than 10 emails before the server fills). But obviously it needs to reset itself when the level drops back down.
So, my quick hack to get that is:
#!/bin/bash
warningpercentage=90 persontomail=me@someplace.com subjectofmail='Warning: You are running low on diskspace!'
oldusage=0 if [ -f ~/.diskspacewarning ]; then oldusage=`cat ~/.diskspacewarning` ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ could be better... how about... oldusage=$(<~/.diskspacewarning) fi
percentageused=$(df -h | sed '1d; s#^.* \([0-9]*\)%.*$#\1#;')
if [ $percentageused -gt $oldusage ]; then if [[ $percentageused -gt $warningpercentage ]]; then ( echo $nicemessage echo df -h ) | mail -s "$subjectofmail" $persontomail fi fi
echo $percentageused > ~/.diskspacewarning
It works, but I'm sure it could be done better? [This is a learning exercise for me!]
There's (obviously) a few more tweaks that could be put in, but hey, it now fits it's requirements, don't break what doesn't need fixing :) Cheers, -- Brett Parker
I recently swapped my computer into a new case - that includes the mainboard, PCI adapters and other peripherals including the hard drive with the existing operating system. The only thing that may have changed is that some PCI adapters *may* be in different slots. If I want the system (Debian) to shutdown it does but the power now won't go off although before the swap it always did. Any ideas? Barry Samuels http://www.beenthere-donethat.org.uk The Unofficial Guide to Great Britain
On Thu, Jan 26, 2006 at 09:22:06PM +0000, Barry Samuels wrote:
I recently swapped my computer into a new case - that includes the mainboard, PCI adapters and other peripherals including the hard drive with the existing operating system.
The only thing that may have changed is that some PCI adapters *may* be in different slots.
If I want the system (Debian) to shutdown it does but the power now won't go off although before the swap it always did.
Any ideas?
The power gets shut down by a control signal from the motherboard to the power suppy I believe, presumably you have a different power supply in the new case, maybe the shutdown signal is absent or different in some way. -- Chris Green (chris@areti.co.uk) "Never ascribe to malice that which can be explained by incompetence."
On 2006.01.27 08:39, Chris Green wrote:
On Thu, Jan 26, 2006 at 09:22:06PM +0000, Barry Samuels wrote:
I recently swapped my computer into a new case - that includes the mainboard, PCI adapters and other peripherals including the hard drive with the existing operating system.
The only thing that may have changed is that some PCI adapters *may* be in different slots.
If I want the system (Debian) to shutdown it does but the power now won't go off although before the swap it always did.
Any ideas?
The power gets shut down by a control signal from the motherboard to the power suppy I believe, presumably you have a different power supply in the new case, maybe the shutdown signal is absent or different in some way.
-- Chris Green (chris@areti.co.uk)
Nope! Same power supply. Barry Samuels http://www.beenthere-donethat.org.uk The Unofficial Guide to Great Britain
On Thu, 2006-01-26 at 21:22 +0000, Barry Samuels wrote:
I recently swapped my computer into a new case - that includes the mainboard, PCI adapters and other peripherals including the hard drive with the existing operating system.
The only thing that may have changed is that some PCI adapters *may* be in different slots.
If I want the system (Debian) to shutdown it does but the power now won't go off although before the swap it always did.
It's most likely an acpi problem..I'd take a look at the kernel boot options to enable/disable these and get the magic combination that works with your hardware. I'd go for noacpi as my first try, just add this to your boot parameters (usually you can interrupt lilo or grub at boot and add these manually) If that doesn't work for you then you could try disabling the apic as well (although it doesn't sound like an apic problem to me. If all else fails try noapic I did have a strange problem with a Compaq machine a while ago whereby as long as a 3com 3c905c was installed it would shutdown and power up again when issued a halt, in that case noapic did the trick for me.
On 2006.01.28 18:55, Wayne Stallwood wrote:
On Thu, 2006-01-26 at 21:22 +0000, Barry Samuels wrote:
I recently swapped my computer into a new case - that includes the mainboard, PCI adapters and other peripherals including the hard drive with the existing operating system.
The only thing that may have changed is that some PCI adapters *may* be in different slots.
If I want the system (Debian) to shutdown it does but the power now won't go off although before the swap it always did.
It's most likely an acpi problem..I'd take a look at the kernel boot options to enable/disable these and get the magic combination that works with your hardware.
Ah! but it always used to work until I swapped the bits into a new case. Nothing else has changed.
I'd go for noacpi as my first try, just add this to your boot parameters (usually you can interrupt lilo or grub at boot and add these manually)
If that doesn't work for you then you could try disabling the apic as well (although it doesn't sound like an apic problem to me. If all else fails try noapic
Could be tricky - this is a dual CPU board. :-)) Barry Samuels http://www.beenthere-donethat.org.uk The Unofficial Guide to Great Britain
On Sat, 2006-01-28 at 19:08 +0000, Barry Samuels wrote:
Ah! but it always used to work until I swapped the bits into a new case. Nothing else has changed.
Ooops sorry I read new case but registered new mainboard for some reason. That is a bit odd, the only thing I can think of is that somehow you have made a slight mistake with the front panel headers and inadvertently shorted a sleep pin or something else that confuses ACPI (you can almost hear the straw clutching)
I'd go for noacpi as my first try, just add this to your boot parameters (usually you can interrupt lilo or grub at boot and add these manually)
If that doesn't work for you then you could try disabling the apic as well (although it doesn't sound like an apic problem to me. If all else fails try noapic
Could be tricky - this is a dual CPU board. :-))
Erm yes that would be a bit of a problem *grin*
On 2006.01.28 19:30, Wayne Stallwood wrote:
On Sat, 2006-01-28 at 19:08 +0000, Barry Samuels wrote:
Ah! but it always used to work until I swapped the bits into a new case. Nothing else has changed.
Ooops sorry I read new case but registered new mainboard for some reason.
That is a bit odd, the only thing I can think of is that somehow you have made a slight mistake with the front panel headers and inadvertently shorted a sleep pin or something else that confuses ACPI
I'll check those again when I next dive into the case.
(you can almost hear the straw clutching)
I'd go for noacpi as my first try, just add this to your boot parameters (usually you can interrupt lilo or grub at boot and add these manually)
If that doesn't work for you then you could try disabling the apic as well (although it doesn't sound like an apic problem to me. If all else fails try noapic
Could be tricky - this is a dual CPU board. :-))
Erm yes that would be a bit of a problem *grin*
Barry Samuels http://www.beenthere-donethat.org.uk The Unofficial Guide to Great Britain
Brett Parker wrote:
On Thu, Jan 26, 2006 at 05:13:37PM +0000, Mark Rogers wrote:
oldusage=`cat ~/.diskspacewarning`
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ could be better... how about... oldusage=$(<~/.diskspacewarning)
Nice, I prefer that. (I've been scripting for years but never had the need and therefore time to play with shell scripts, which I really *should* be much better at).
There's (obviously) a few more tweaks that could be put in, but hey, it now fits it's requirements, don't break what doesn't need fixing :)
If it ain't broke, keep fixing it until it is! Basically, I'm trying to learn from this so keeping it as it is just because it works may lead me to bad habits. Anyway, I now have some hysteresis issues; level hovering around 95% leads to a level at 95%/94%/95%/94%/etc, and triggers an email each time it goes back to 95%. Haven't decided yet the best way to handle that, so suggestions welcomed! -- Mark Rogers More Solutions Ltd :: 0845 45 89 555
On Fri, Jan 27, 2006 at 10:50:03AM +0000, Mark Rogers wrote:
Anyway, I now have some hysteresis issues; level hovering around 95% leads to a level at 95%/94%/95%/94%/etc, and triggers an email each time it goes back to 95%. Haven't decided yet the best way to handle that, so suggestions welcomed!
Time to buy a *much* bigger hard disk ;) Thanks Adam -- jabberid = quinophex@jabber.earth.li AFFS || http://www.affs.org.uk/ || Not a filesystem
Adam Bower wrote:
Time to buy a *much* bigger hard disk ;)
Thanks! Actually I have plenty of disk space spare (just not on this virtual server), and although I can shuffle things around I'd rather find the cause of the problem. In this case I had a "web" mailbox collecting bounced emails for the last few years unnoticed (Apache runs as "web"). It had accrued 400MB of mail (on a 2GB server that's fairly substantial). It's actually hovering around 85% full at the moment, and I reckon that it should be closer to 60% if I could spare the time to find all the junk that's accumulated. I have the threshold for the email warning set much lower so that I can test it. I'm running the script every 30 mins and getting one warning every hour or two, which I'm pretty sure is down to hysteresis. Can I do maths within my script? Eg only if the current %age used value is at least 2% lower than the stored value do I overwrite the stored value, so 85/84/85/84 readings cause 85 to get stored as last reading, but 84 doesn't overwrite it so the subsequent 85 doesn't exceed it and generate an email. (Did that make any sense at all?) -- Mark Rogers More Solutions Ltd :: 0845 45 89 555
On Mon, 2006-01-30 at 08:23 +0000, Mark Rogers wrote:
Can I do maths within my script? Eg only if the current %age used value is at least 2% lower than the stored value do I overwrite the stored value, so 85/84/85/84 readings cause 85 to get stored as last reading, but 84 doesn't overwrite it so the subsequent 85 doesn't exceed it and generate an email. (Did that make any sense at all?)
Try a man bc or perhaps man dc if reverse-polish notation doesn't cause your brain to keep segfaulting.
Wayne Stallwood wrote:
Try a man bc or perhaps man dc if reverse-polish notation doesn't cause your brain to keep segfaulting.
Ta, should have thought of that myself. RP notation is fun! I've still got my old Sinclair calculator which uses it. But I'll stick with bc :-) -- Mark Rogers More Solutions Ltd :: 0845 45 89 555
If I want to run a script on my externally hosted server to mark me if my disk space is getting low, what's the simplest way to do it?
Say I want a report if "df" reports more than 90% used, I want the script to run regularly but not send an email every time it runs if the problem hasn't worsend, and want to include the output from 'du -h' in the email?
Then, once I have that sorted, what's the "best" way to do this kind of thing?
Oh, and what's the best way to work out *why* I only have 1% left on my server?
-- Mark Rogers More Solutions Ltd :: 0845 45 89 555
_______________________________________________ main@lists.alug.org.uk http://www.alug.org.uk/ http://lists.alug.org.uk/mailman/listinfo/main Unsubscribe? See message headers or the web site above!
df | awk '/dev/ && $3 > 0 && ($3 / $2 * 100) > 85 { print $6,"is",$5,"full [",$3,"/",$2,"=",$4,"free]" } ' | mail -s $subjectofmail $persontomail Cheers Stuart -- The problem with the world is stupidity. Not saying there should be a capital punishment for stupidity, but why don't we just take the safety labels off of everything and let the problem solve itself?
Depends...if you have root access to the external server Why not use the disk quota system (probably already built into your kernel) it has the facility to generate warnings and stop writes at different levels of fullness. The only time I used it I was setting quotas for individual groups but I am sure you could set it up as a wildcard group with ease. Looking back at my notes from about 5 years ago when I did this, I had done the following Compiled the quota support into the kernel (probably already there nowadays) ran /sbin/quotacheck -avug and /sbin/quotaon -avug at startup as part of my init scripts. then there are a couple of mods to fstab and a user.quota and group.quota. Then I had this script (sorry about the state of this, it was an early point in my days of mucking about with linux) Watch the wrapping. #! /bin/bash # Wayne Stallwood 2001 # A little script to check the status of quota's on a system and mail somebody if the soft quota is breached # Add details of each group/user to the bottom of this file as directed by the comments below # You also need to make sure that quotainit.sc is run at system startup ,that the filesystems concerned # Are mounted with either the usrquota or grpquota tags in fstab and the user.quota and group.quota files have been created HEADER=/home/wayne/bailheader.conf TODAY=$(date +%Y%m%d) OTHERDAY=$(date --date '6 days') function telltale () { echo 'The' \"$@\" 'Group on' $HOSTNAME 'have exceeded their allocated space' >> /home/wayne/output.msg echo 'Please Conact Wayne or The CVS Admin to Beg for some more'
/home/wayne/output.msg echo 'or to get someting old removed,' >> /home/wayne/output.msg echo 'Failure to do this may result in the' \"$@\" 'Group being unable to commit' >> /home/wayne/output.msg echo 'anything else into CVS from' $OTHERDAY 'Or even before if they keep adding stuff !' >> /home/wayne/output.msg less $HEADER >> /home/wayne/output.msg /usr/sbin/sendmail $MAILUSER < /home/wayne/output.msg rm /home/wayne/output.msg touch '/root/'$@'havebeentold'
} function allfine () { echo 'The' \"$@\" 'Group on' $HOSTNAME 'have reduced their used space'
/home/wayne/output.msg echo 'The previous warnings can now be ignored' /home/wayne/output.msg less $HEADER >> /home/wayne/output.msg /usr/sbin/sendmail $MAILUSER < /home/wayne/output.msg rm /home/wayne/output.msg rm -f '/root/'$@'havebeentold' }
function checkit () { if [ -e '/root/'$@'havebeentold' ] then quota -u $GROUPNAME | grep "*" || allfine $@ else quota -u $GROUPNAME | grep "*" && telltale $@ fi } # Add user and group quotas to check here as follows # GROUPNAME=type name of group / user you want to check here as it appears in the passwd file # MAILSUER=type email address of the person you want notified here, multiple addresses may be delimited by a space # If the group name is the same as the mail user you could define MAILUSER=$GROUPNAME'@domain.co.uk' just below # these comments # checkit 'type visible name of group here eg artists, programmers etc' # You can have as many of these as you like, but you must insure that each checkit name is proceeded by a GROUPNAME # and a MAILUSER Otherwise nasty things will happen GROUPNAME=wayne MAILUSER=wayne@cyberlife.co.uk checkit ITdept
participants (9)
-
Adam Bower -
Barry Samuels -
Barry Samuels -
Brett Parker -
Chris Green -
Mark Rogers -
Stuart Fox -
Ted.Harding＠nessie.mcc.ac.uk -
Wayne Stallwood