Hi Folks,
On one (A) of my 3 running machines, I noticed that 'ls -ls' on /var/log shows:
A: 44 -r-------- 1 root root 19136220 Oct 30 10:52 lastlog
which is inconsistent: 44K according to $1, but 19136220 bytes (19MB) according to $6. The latter is enormous! However, it seems that it has some validity, since
# cat lastlog | wc -c 19136220
gives the same result, here obtained by byte-counting the output of 'cat'. This first got me wondering if the lastlog file or its inode contents were corrupted.
(For reference below: I don't have a file /var/log/faillog on this particular machine, A).
Interestingly, however, when I look at 'ls -ls' for my two other machines (B & C), I get
B: 3 -rw------- 1 root root 12216 Nov 6 15:32 faillog 3 -rw-r--r-- 1 root root 16128 Nov 6 15:32 lastlog
C: 8 -rw------- 1 root root 12072 Nov 4 13:17 faillog 16 -rw-r--r-- 1 root tty 146876 Nov 4 13:17 lastlog
and for both machines, and also for both 'faillog' and 'lastlog', I see the same sort of discrepancy between $1 (size in KB) and $6 (size in bytes), the latter being distinctly too big compared with the former.
I do not see this discrepancy for any other files in /var/log, on any of the three machines.
This suggests that there's something a bit weird about 'lastlog' and 'faillog' as files, and that this is deliberate! And I'm wondering how on earth a 19MB lastlog file could arise (this machine, A, has been in use for a shorter time (about 1 year) than C (about 3.5 years) and much shorter than B (about 7 years); yet the sizes of 'lastlog' are in the reverse order!)
The output of the 'lastlog' *command* in all cases is just a few dozen lines; but, again interestingly, doing it on machine (A) (the one with the 19MB lastlog) induces semi-paralysis and much swapping for a while.
I'm puzzled! Comments welcome.
Best wishes to all, Ted.
-------------------------------------------------------------------- E-Mail: (Ted Harding) Ted.Harding@nessie.mcc.ac.uk Fax-to-email: +44 (0)870 094 0861 [NB: New number!] Date: 07-Nov-04 Time: 11:15:56 ------------------------------ XFMail ------------------------------
On Sun, 07 Nov 2004 11:20:16 +0100 (GMT), Ted Harding ted.harding@nessie.mcc.ac.uk wrote:
Hi Folks,
On one (A) of my 3 running machines, I noticed that 'ls -ls' on /var/log shows:
A: 44 -r-------- 1 root root 19136220 Oct 30 10:52 lastlog
which is inconsistent: 44K according to $1, but 19136220 bytes (19MB) according to $6.
"ls -s" shows the file sizes in blocks. The question is what is a block?
My log directory has a file using more blocks than would appear necessary to hold all the bytes: 108 -rw-r--r-- 1 root root 102591 Jan 1 1996 installer.log.1
Anyway, it is not even as if lastlog is open by another process - I checked with lsof.
An interesting mystery!
Tim.
* Ted Harding (Ted.Harding@nessie.mcc.ac.uk) wrote:
Hi Folks,
On one (A) of my 3 running machines, I noticed that 'ls -ls' on /var/log shows:
A: 44 -r-------- 1 root root 19136220 Oct 30 10:52 lastlog
which is inconsistent: 44K according to $1, but 19136220 bytes (19MB) according to $6. The latter is enormous! However, it seems that it has some validity, since
# cat lastlog | wc -c 19136220
gives the same result, here obtained by byte-counting the output of 'cat'. This first got me wondering if the lastlog file or its inode contents were corrupted.
We have here a sparse file; one of the more useful but slightly confusing features of Unix. If you start with an empty file, 'seek' a few GB into the file and write one byte you'll actually only store one block of data for the file (oh and a handful of other data saying where it is). Now in this case the empty space normally reads as zero; normal programs just read these zeros without knowing that there is anything special about the file (and so if you copy a sparse file without doing special stuff you suddenly increase the disc usage!).
Now what lastlog does is that it stores a block of data for each user of 292 bytes in length (struct lastlog); and it stores those at offset: uid * sizeof(struct lastlog)
in the file.
B: 3 -rw------- 1 root root 12216 Nov 6 15:32 faillog 3 -rw-r--r-- 1 root root 16128 Nov 6 15:32 lastlog
Did you ever log into that other than as root? Or perhaps you have a user with uid 54?
C: 8 -rw------- 1 root root 12072 Nov 4 13:17 faillog 16 -rw-r--r-- 1 root tty 146876 Nov 4 13:17 lastlog
I'm guessing you have a user on here with uid 502? (146876/292).
So now we come to the monster file; 19136220 / 292=65535 that is odd. That suggests that there is an entry in lastlog for user 65534 or 65535 - have you ever logged onto the machine as 'nobody'?
Dave -----Open up your eyes, open up your mind, open up your code ------- / Dr. David Alan Gilbert | Running GNU/Linux on Alpha,68K| Happy \ \ gro.gilbert @ treblig.org | MIPS,x86,ARM,SPARC,PPC & HPPA | In Hex / \ _________________________|_____ http://www.treblig.org |_______/
On 07-Nov-04 Dr. David Alan Gilbert wrote:
- Ted Harding (Ted.Harding@nessie.mcc.ac.uk) wrote:
Hi Folks,
On one (A) of my 3 running machines, I noticed that 'ls -ls' on /var/log shows:
A: 44 -r-------- 1 root root 19136220 Oct 30 10:52 lastlog
[...] We have here a sparse file; one of the more useful but slightly confusing features of Unix. If you start with an empty file, 'seek' a few GB into the file and write one byte you'll actually only store one block of data for the file (oh and a handful of other data saying where it is). Now in this case the empty space normally reads as zero; normal programs just read these zeros without knowing that there is anything special about the file (and so if you copy a sparse file without doing special stuff you suddenly increase the disc usage!).
Now what lastlog does is that it stores a block of data for each user of 292 bytes in length (struct lastlog); and it stores those at offset: uid * sizeof(struct lastlog)
in the file.
Dave, thanks for an illuminating explanation and especially the pointer to the offset calculation, and how the file size depends on the largest uid that has logged in! Most helpful and useful.
B: 3 -rw------- 1 root root 12216 Nov 6 15:32 faillog 3 -rw-r--r-- 1 root root 16128 Nov 6 15:32 lastlog
Did you ever log into that other than as root? Or perhaps you have a user with uid 54?
Hmmm -- I have uids in /etc/passwd in the range 500-508, and I regularly log in with uids 500 and 503.
C: 8 -rw------- 1 root root 12072 Nov 4 13:17 faillog 16 -rw-r--r-- 1 root tty 146876 Nov 4 13:17 lastlog
I'm guessing you have a user on here with uid 502? (146876/292).
Here 500-502, so it's consistent!
So now we come to the monster file; 19136220 / 292=65535 that is odd. That suggests that there is an entry in lastlog for user 65534 or 65535 - have you ever logged onto the machine as 'nobody'?
User "nobody" has uid 99 on this machine. However, there is an "nfsnobody" with uid 65534, but to my memory I've not logged in as such on this machine. However, it may be that some process started up by the machine itself (Red Hat 9) does this. I've certainly hooked the machine up to nfs many times.
However, the 'lastlog' program gives: nobody **Never logged in** [...] nfsnobody **Never logged in**
Anyway, all very interesting! Thanks.
Ted.
-------------------------------------------------------------------- E-Mail: (Ted Harding) Ted.Harding@nessie.mcc.ac.uk Fax-to-email: +44 (0)870 094 0861 [NB: New number!] Date: 07-Nov-04 Time: 14:05:33 ------------------------------ XFMail ------------------------------
On 07-Nov-04 Ted Harding wrote:
Hi Folks,
On one (A) of my 3 running machines, I noticed that 'ls -ls' on /var/log shows:
A: 44 -r-------- 1 root root 19136220 Oct 30 10:52 lastlog
which is inconsistent: 44K according to $1, but 19136220 bytes (19MB) according to $6. The latter is enormous! However, it seems that it has some validity, since
# cat lastlog | wc -c 19136220
gives the same result, here obtained by byte-counting the output of 'cat'.
I've picked up a thread on the debian-user list which deals with this:
http://lists.debian.org/debian-user/2000/05/msg01593.html
(follow it through to the end with "Next in Thread"). The respondent to the queries and comments is Miquel van Smoorenburg, who is certainly an authority on filesystems!
At the end, MvS points to the following explanation of the mechanism which is operating here:
http://www.nic.com/~cheah/hole.html
Anyway, the upshot is that the 44K in $1 would be the size of the file as stored on the hard drive, while the 19136220 in $6 is the size of the output when this file is read (the kernel fills in megabytes of nulls which do not get stored on disk).
But I still wonder why it needs to be so big ...
Best wishes to all, Ted.
-------------------------------------------------------------------- E-Mail: (Ted Harding) Ted.Harding@nessie.mcc.ac.uk Fax-to-email: +44 (0)870 094 0861 [NB: New number!] Date: 07-Nov-04 Time: 12:53:14 ------------------------------ XFMail ------------------------------
-
Dr. David Alan Gilbert -
Ted.Harding＠nessie.mcc.ac.uk -
Tim Green