I'm having a problem with my system and I can't see where the solution is despite looking online. Audacity said that the disc was short of space and just sat there doing nothing. So I killed the task and looked at the space with 'df'. It shows that /dev/sdb5 has zero space left. Here's the relevant output from df :- Filesystem Size Used Avail Use% Mounted on devtmpfs 3.0G 0 3.0G 0% /dev tmpfs 3.0G 1.4M 3.0G 1% /dev/shm tmpfs 3.0G 940K 3.0G 1% /run /dev/sdb5 7.6G 7.2G 0 100% / /dev/sdb7 7.6G 5.6G 1.7G 78% /usr tmpfs 3.0G 0 3.0G 0% /sys/fs/cgroup tmpfs 3.0G 88K 3.0G 1% /tmp /dev/sdb8 849G 202G 647G 24% /home tmpfs 597M 12K 597M 1% /run/user/1000 So where has the 7.6 gigs worth of space gone on sdb5 or am I looking in the wrong place ? -- __ __| |_ __ __ .----------------------------------------------. / _/ _` \ V V / | mailto:alug_cdw@the-walker-household.co.uk | \__\__,_|\_/\_/ |______________________________________________|
On 03/01/16 11:23, Chris Walker wrote:
I'm having a problem with my system and I can't see where the solution is despite looking online.
Audacity said that the disc was short of space and just sat there doing nothing. So I killed the task and looked at the space with 'df'.
It shows that /dev/sdb5 has zero space left. Here's the relevant output from df :-
Filesystem Size Used Avail Use% Mounted on devtmpfs 3.0G 0 3.0G 0% /dev tmpfs 3.0G 1.4M 3.0G 1% /dev/shm tmpfs 3.0G 940K 3.0G 1% /run /dev/sdb5 7.6G 7.2G 0 100% / /dev/sdb7 7.6G 5.6G 1.7G 78% /usr tmpfs 3.0G 0 3.0G 0% /sys/fs/cgroup tmpfs 3.0G 88K 3.0G 1% /tmp /dev/sdb8 849G 202G 647G 24% /home tmpfs 597M 12K 597M 1% /run/user/1000
So where has the 7.6 gigs worth of space gone on sdb5 or am I looking in the wrong place ?
try du -x -d 2 / as superuser as a start to what has used the space. Nev
On Sun, 3 Jan 2016 11:39:26 +0000 nev young <crw_1622@hotmail.com> wrote:
On 03/01/16 11:23, Chris Walker wrote:
I'm having a problem with my system and I can't see where the solution is despite looking online.
Audacity said that the disc was short of space and just sat there doing nothing. So I killed the task and looked at the space with 'df'.
It shows that /dev/sdb5 has zero space left. Here's the relevant output from df :-
Filesystem Size Used Avail Use% Mounted on devtmpfs 3.0G 0 3.0G 0% /dev tmpfs 3.0G 1.4M 3.0G 1% /dev/shm tmpfs 3.0G 940K 3.0G 1% /run /dev/sdb5 7.6G 7.2G 0 100% / /dev/sdb7 7.6G 5.6G 1.7G 78% /usr tmpfs 3.0G 0 3.0G 0% /sys/fs/cgroup tmpfs 3.0G 88K 3.0G 1% /tmp /dev/sdb8 849G 202G 647G 24% /home tmpfs 597M 12K 597M 1% /run/user/1000
So where has the 7.6 gigs worth of space gone on sdb5 or am I looking in the wrong place ?
try du -x -d 2 / as superuser as a start to what has used the space.
That produced a long list of stuff so I referred to this page - http://www.cyberciti.biz/datacenter/linux-unix-bsd-osx-cannot-write-to-hard-... Looking at item 4 cleaned that up but still produced nothing of any use to me (it had things from a NAS drive in there too). So having deleted several kernel related items, I've left it there until the next time. I've kept both the emails though for future reference and to avoid having to ask the same question again. -- __ __| |_ __ __ .----------------------------------------------. / _/ _` \ V V / | mailto:alug_cdw@the-walker-household.co.uk | \__\__,_|\_/\_/ |______________________________________________|
On 03/01/16 11:23, Chris Walker wrote:
I'm having a problem with my system and I can't see where the solution is despite looking online.
Audacity said that the disc was short of space and just sat there doing nothing. So I killed the task and looked at the space with 'df'.
It shows that /dev/sdb5 has zero space left. Here's the relevant output from df :-
Filesystem Size Used Avail Use% Mounted on devtmpfs 3.0G 0 3.0G 0% /dev tmpfs 3.0G 1.4M 3.0G 1% /dev/shm tmpfs 3.0G 940K 3.0G 1% /run /dev/sdb5 7.6G 7.2G 0 100% / /dev/sdb7 7.6G 5.6G 1.7G 78% /usr tmpfs 3.0G 0 3.0G 0% /sys/fs/cgroup tmpfs 3.0G 88K 3.0G 1% /tmp /dev/sdb8 849G 202G 647G 24% /home tmpfs 597M 12K 597M 1% /run/user/1000
So where has the 7.6 gigs worth of space gone on sdb5 or am I looking in the wrong place ?
if I do ls /, I get something like bin dev mnt root sys boot etc lib run tmp home lost+found opt sbin usr media proc srv var Some of those are temporary file systems. Some you have mounted as separate "drives". Everything else will be using up space on "\" partition. However, having a machine with limited disk space myself, my money is on old kernels. Unless you've deleted them after updating, they sit there taking up space. The latest version of Grub tends to hide how many old kernels you have installed from you. IF you're running ububtu, try installing ubuntutweak (I'll leave that as an excercise). Then run its janitor. Failing that, OLD SKOOL method is to uninstall them manually. I think files you need start with "linux-image" but be careful. DO NOT DELETE THE CURRENT KERNEL. Try just deleting one old one (lower version numbers) Backup first Be careful Don't do it if you don't know what you're doing Caveat Emptor etc Good luck Steve
On Sun, 3 Jan 2016 12:13:26 +0000 steve-ALUG@hst.me.uk wrote:
On 03/01/16 11:23, Chris Walker wrote:
I'm having a problem with my system and I can't see where the solution is despite looking online.
[snip]
So where has the 7.6 gigs worth of space gone on sdb5 or am I looking in the wrong place ?
[snip] Some of those are temporary file systems. Some you have mounted as separate "drives". Everything else will be using up space on "\" partition. However, having a machine with limited disk space myself, my money is on old kernels. Unless you've deleted them after updating, they sit there taking up space. The latest version of Grub tends to hide how many old kernels you have installed from you.
I tend to remove the oldest entry from the grub menu almost as soon as a new kernel is installed so I'd forgotten about them staying on the disc somewhere.
IF you're running ububtu, try installing ubuntutweak (I'll leave that as an excercise). Then run its janitor. Failing that, OLD SKOOL method is to uninstall them manually. I think files you need start with "linux-image" but be careful. DO NOT DELETE THE CURRENT KERNEL. Try just deleting one old one (lower version numbers)
I'm not running ubuntu, I can't get on with it. I run Mageia.
Backup first
I have a regular routine to do that having been bitten by finger trouble in the past.
Be careful Don't do it if you don't know what you're doing
Not sure about that bit :-)
Good luck
Who needs luck when you have a backup? :-) If I look in /boot I see lots of symvers, System.map and vmlinuz files, lots of which refer to v3 kernels and I'm now running v4.1.13 so I've deleted the old stuff having made sure that those files are present in the backup. I now over 60MB free on the drive and all is running smoothly again. Thanks for your help. -- __ __| |_ __ __ .----------------------------------------------. / _/ _` \ V V / | mailto:alug_cdw@the-walker-household.co.uk | \__\__,_|\_/\_/ |______________________________________________|
On Sun, 3 Jan 2016 12:13:26 +0000 steve-ALUG@hst.me.uk wrote:
On 03/01/16 11:23, Chris Walker wrote:
I'm having a problem with my system and I can't see where the solution is despite looking online.
Backup first
I'm doing an rsync to another 1TB disc with --verbose on and I thought it was taking a long time. It's trying to rsync /proc/kcore and it's running at 100MB/s and it says it's done 0%. That puzzled me so I've just looked at the file. The file manager (Dolphin) says that it's 128TB in size. That's pretty good seeing as it's on a 1TB disc. So should I let it run to see if it sorts itself out or bin it off? What do I then do about the kcore file?
Be careful Don't do it if you don't know what you're doing
I'm at that stage now! -- __ __| |_ __ __ .----------------------------------------------. / _/ _` \ V V / | mailto:alug_cdw@the-walker-household.co.uk | \__\__,_|\_/\_/ |______________________________________________|
On 12/01/2016 19:03, Chris Walker wrote:
it was taking a long time. It's trying to rsync /proc/kcore and it's
/proc is a virtual filesystem - no real files, just a "filesystem-like" view into the current state of the system. For example, /proc/meminfo is a summary of the state of the system's memory usage, or /proc/423/cmdline is the command line options for process 423, or /proc/fs/ext4/mmcblk0p2/options is a list of the current mount options of ext4 filesystem on partition 2 of device mmcblk0p2 And /proc/kcore is a look into the current kernel addressable memory space. Some of it will be real memory, most won't. So, you don't need to and shouldn't back up anything in /proc Or for that matter some of the other virtual/temporary file systems, e.g. /sys or possibly /tmp. Bill -- Red to red, black to black, switch it on, but stand well back.
On 12/01/16 20:05, Bill Hill wrote:
On 12/01/2016 19:03, Chris Walker wrote:
it was taking a long time. It's trying to rsync /proc/kcore and it's /proc is a virtual filesystem - no real files, just a "filesystem-like" view into the current state of the system. For example, /proc/meminfo is a summary of the state of the system's memory usage, or /proc/423/cmdline is the command line options for process 423, or /proc/fs/ext4/mmcblk0p2/options is a list of the current mount options of ext4 filesystem on partition 2 of device mmcblk0p2 And /proc/kcore is a look into the current kernel addressable memory space. Some of it will be real memory, most won't. So, you don't need to and shouldn't back up anything in /proc Or for that matter some of the other virtual/temporary file systems, e.g. /sys or possibly /tmp.
Bill
You know that list that started this: Filesystem Size Used Avail Use% Mounted on devtmpfs 3.0G 0 3.0G 0% /dev tmpfs 3.0G 1.4M 3.0G 1% /dev/shm tmpfs 3.0G 940K 3.0G 1% /run /dev/sdb5 7.6G 7.2G 0 100% / /dev/sdb7 7.6G 5.6G 1.7G 78% /usr tmpfs 3.0G 0 3.0G 0% /sys/fs/cgroup tmpfs 3.0G 88K 3.0G 1% /tmp /dev/sdb8 849G 202G 647G 24% /home tmpfs 597M 12K 597M 1% /run/user/1000 A quick rule (that I've just invented) is probably back up anything that has a file system tmpfs or devtmpfs. In my rsync backup job I have --exclude-from=excludefilelist excludefilelist contains (amongst other things) /sys/* /dev/* /proc/* /media/* /mnt/* /tmp/* /lost+found/* /home/MAINUSER/.gvfs/* /home/MAINUSER/.gvfs /var/tmp/* /root/.Trash/* /root/.local/share/Trash/* *.iso home/MAINUSER/.local/share/Trash/* home/\\*/.local/share/Trash/* /home/\\*/.gvfs/* /exports/* sys, dev, proc, aren't usually backed up media and mnt are mount points for external media/drives tmp and /var/tmp are temporary files - generally you don't need to backup these files Replace MAINUSER with the user you're logged on with when you backup on the .gvfs lines - I had some problems with these once upon a time. It's something to do with another temporary mounted file system I think. Do you really want to back up the trash? I exclude ISO files, as I have some CD images stored on my drive. I don't need them backed up. I could just download them again. I exclude /exports because I have some nfs4 file exports. These basically link/map somewhere on the disk back into here. If you back up this, it will basically backup some directories twice. Be careful if you back the disk somewhere on the same disk. (Actually, in Linux, you probably have to be careful about this almost every time). Imagine you are backing up all files on /. You are backing up to /mnt/externaldrive. /mnt/externaldrive is a subdirectory of /. If you back up / it will back up to /mnt/externaldrive that will begin to fill up. Eventually as the backup proceeds, it will start backing up /mnt/externaldrive. This directory includes all the files that it's backed up already, so it backs them up. As it does this, there are more files in /mnt/externaldrive, which it backs up, until your disk fills up. The solution is to exclude the mount point for your backup drive, or all mount points (as I have done). I hope that gives you some pointers Steve
On Tue, 12 Jan 2016 23:01:33 +0000 steve-ALUG@hst.me.uk wrote:
On 12/01/16 20:05, Bill Hill wrote:
On 12/01/2016 19:03, Chris Walker wrote:
it was taking a long time. It's trying to rsync /proc/kcore and it's /proc is a virtual filesystem - no real files, just a "filesystem-like" view into the current state of the system. For example, /proc/meminfo is a summary of the state of the system's memory usage, or /proc/423/cmdline is the command line options for process 423, or /proc/fs/ext4/mmcblk0p2/options is a list of the current mount options of ext4 filesystem on partition 2 of device mmcblk0p2 And /proc/kcore is a look into the current kernel addressable memory space. Some of it will be real memory, most won't. So, you don't need to and shouldn't back up anything in /proc Or for that matter some of the other virtual/temporary file systems, e.g. /sys or possibly /tmp.
Bill
You know that list that started this:
It's because of that and the continuing problems that I need an up-to-date backup.
Filesystem Size Used Avail Use% Mounted on devtmpfs 3.0G 0 3.0G 0% /dev tmpfs 3.0G 1.4M 3.0G 1% /dev/shm tmpfs 3.0G 940K 3.0G 1% /run /dev/sdb5 7.6G 7.2G 0 100% / /dev/sdb7 7.6G 5.6G 1.7G 78% /usr tmpfs 3.0G 0 3.0G 0% /sys/fs/cgroup tmpfs 3.0G 88K 3.0G 1% /tmp /dev/sdb8 849G 202G 647G 24% /home tmpfs 597M 12K 597M 1% /run/user/1000
A quick rule (that I've just invented) is probably back up anything that has a file system tmpfs or devtmpfs.
In my rsync backup job I have --exclude-from=excludefilelist
excludefilelist contains (amongst other things) /sys/* /dev/* /proc/* /media/* /mnt/* /tmp/* /lost+found/* /home/MAINUSER/.gvfs/* /home/MAINUSER/.gvfs /var/tmp/* /root/.Trash/* /root/.local/share/Trash/* *.iso home/MAINUSER/.local/share/Trash/* home/\\*/.local/share/Trash/* /home/\\*/.gvfs/* /exports/*
I've created a script to do the rsync stuff and have added those to the exclude list. [snip]
Be careful if you back the disk somewhere on the same disk. (Actually, in Linux, you probably have to be careful about this almost every time).
It's being backed up to a 1TB disc which is fixed in the machine and is used as a transit storage between the linux side and Windows 10 (the machine dual boots).
Imagine you are backing up all files on /. You are backing up to /mnt/externaldrive. /mnt/externaldrive is a subdirectory of /. If you back up / it will back up to /mnt/externaldrive that will begin to fill up. Eventually as the backup proceeds, it will start backing up /mnt/externaldrive. This directory includes all the files that it's backed up already, so it backs them up. As it does this, there are more files in /mnt/externaldrive, which it backs up, until your disk fills up.
The solution is to exclude the mount point for your backup drive, or all mount points (as I have done).
The drive to which I'm doing the backup is mounted as /media/Disc_Swap_Space/ so /media/ was already on the exclude list. I also have a NAS drive which is also mounted at /media/. I've fallen at that hurdle before!
I hope that gives you some pointers
Indeed it does. Thanks too to Bill Hill for his ideas. Next question! I want to wipe everything from this drive and re-install with the latest release of Mageia 5. I also want to repartition the disc to avoid running out of space in the future. Is there a good guide somewhere that people would advise me to look at before doing that? I realise that I can just search for something but I want a guide that people think is the definitive guide and one that can be trusted for good advice. -- __ __| |_ __ __ .----------------------------------------------. / _/ _` \ V V / | mailto:alug_cdw@the-walker-household.co.uk | \__\__,_|\_/\_/ |______________________________________________|
On 13/01/16 12:04, Chris Walker wrote:
Next question! I want to wipe everything from this drive and re-install with the latest release of Mageia 5. I also want to repartition the disc to avoid running out of space in the future. Is there a good guide somewhere that people would advise me to look at before doing that? I realise that I can just search for something but I want a guide that people think is the definitive guide and one that can be trusted for good advice.
Definitive guide? Ha! That's a good one! :-) I think you'll have to google and find various opinions! Good luck! When I started, I just had everything in / I then have progressed to having / and /home. I think some people have /, /home, /var and /tmp, possibly more. The reason for more partitions is, say, some rogue process or interenet attack filling up a log file (say) - if the log files are in /var and that's in a separate partition, then just that partition will fill up, and hopefully your system will fail gracefully, or not fail at all. However, you've got to balance that with the hassle of having multiple partitions. For instance, you won't run into the "running out of space" problem if everything's stored under /, unless something fills up the entire disk. However, if this does happen, then you're in trouble, I haven't gone down the "loads of partitions" route because I don't know how to apportion the disk space. For instance, if you have a /, /home, /var and /tmp, how much space should you allocate to each. What do you do if you get it wrong? This is especially hard for me as my setup also has RAID, which adds another level of complexity. I *think* that you can use something called LVM (I think) which (possibly) stands for Logical Volume Manager to create partions that can be easily resized. Otherwise, I don't know how you can resize your partitions without starting from scratch - perhaps it can be done, I don't know! I don't know how magea works. However, there may be some mileage in this (which I think is the approach that Mint takes. Generate a list of installed packages (your package manager should be able to do this - (apt can I think, so can synaptic). Copy all the stuff you need, esp you /home directories and /etc for all the config files. Format and repartition, install, then run the list of packages that were installed through your package manager to ensure that they're reinstalled. Tweak the config files, restore /home. That may work in theory, but I've not tried it. In Ubuntu for example, some packages get obsoleted and replaced by others. I don't know how the above approach would cope with that. Good luck. Steve
Partitioning is a religious issue, but after more than 30 years supporting Unix & Unix-related systems, for my personal systems I put /home on one partition and everything else on a separate one. In these days of cheap terabyte disks, filling up is rarely an issue, and splitting /home off makes upgrades a doodle. Better still, I put /home on one disk and everything else on another, so when I upgrade, I just swap the system disk for a new (usually bigger) one, and if I need to retrieve anything from the old system, I can just plug the disk into a USB caddy. Note that this is very different from what we do at work! Huge. On 13 Jan 2016, at 14:38, steve-ALUG@hst.me.uk wrote:
On 13/01/16 12:04, Chris Walker wrote:
Next question! I want to wipe everything from this drive and re-install with the latest release of Mageia 5. I also want to repartition the disc to avoid running out of space in the future. Is there a good guide somewhere that people would advise me to look at before doing that? I realise that I can just search for something but I want a guide that people think is the definitive guide and one that can be trusted for good advice.
Definitive guide? Ha! That's a good one! :-) I think you'll have to google and find various opinions! Good luck!
When I started, I just had everything in / I then have progressed to having / and /home. I think some people have /, /home, /var and /tmp, possibly more.
The reason for more partitions is, say, some rogue process or interenet attack filling up a log file (say) - if the log files are in /var and that's in a separate partition, then just that partition will fill up, and hopefully your system will fail gracefully, or not fail at all. However, you've got to balance that with the hassle of having multiple partitions.
For instance, you won't run into the "running out of space" problem if everything's stored under /, unless something fills up the entire disk. However, if this does happen, then you're in trouble,
I haven't gone down the "loads of partitions" route because I don't know how to apportion the disk space. For instance, if you have a /, /home, /var and /tmp, how much space should you allocate to each. What do you do if you get it wrong? This is especially hard for me as my setup also has RAID, which adds another level of complexity. I *think* that you can use something called LVM (I think) which (possibly) stands for Logical Volume Manager to create partions that can be easily resized. Otherwise, I don't know how you can resize your partitions without starting from scratch - perhaps it can be done, I don't know!
I don't know how magea works. However, there may be some mileage in this (which I think is the approach that Mint takes. Generate a list of installed packages (your package manager should be able to do this - (apt can I think, so can synaptic). Copy all the stuff you need, esp you /home directories and /etc for all the config files. Format and repartition, install, then run the list of packages that were installed through your package manager to ensure that they're reinstalled. Tweak the config files, restore /home.
That may work in theory, but I've not tried it. In Ubuntu for example, some packages get obsoleted and replaced by others. I don't know how the above approach would cope with that.
Good luck.
Steve
_______________________________________________ main@lists.alug.org.uk http://www.alug.org.uk/ http://lists.alug.org.uk/mailman/listinfo/main Unsubscribe? See message headers or the web site above!
— Sent from my Psion 5MX
participants (5)
-
Bill Hill -
Chris Walker -
Huge -
nev young -
steve-ALUG＠hst.me.uk