I have a log file which looks like this: 20110905-000107 803 cmd1853 - Host:/127.0.0.1 opened command interface from port: 54365 20110905-000108 761 cmd1854 - Command: 20110905-000108 766 cmd1853 - Response: 201 Service ready 20110905-000108 766 cmd1854 - Response: 201 Service ready 20110905-000108 803 cmd1854 - Host:/127.0.0.1 opened command interface from port: 57052 20110905-000108 804 cmd1853 - Host:/127.0.0.1 closed command interface from port: 54365 20110905-000113 804 cmd1854 - Host:/127.0.0.1 closed command interface from port: 57052 20110905-000313 761 cmd1858 - Command: 20110905-000313 766 cmd1856 - Response: 201 Service ready 20110905-000313 766 cmd1858 - Response: 201 Service ready
I want to sort it in numeric order on the cmd value. Note that the cmd number isn't always 4 digits and I want cmd9 to appear before cmd10, etc.
Can I do this with sort?
I've seen examples that suggest that something along the lines of sort -n -k3.4 .. would do it but I can't get it working in any of the various permutations I've tried.
On 06-Sep-11 12:35:57, Mark Rogers wrote:
I have a log file which looks like this: 20110905-000107 803 cmd1853 - Host:/127.0.0.1 opened command interface from port: 54365 20110905-000108 761 cmd1854 - Command: 20110905-000108 766 cmd1853 - Response: 201 Service ready 20110905-000108 766 cmd1854 - Response: 201 Service ready 20110905-000108 803 cmd1854 - Host:/127.0.0.1 opened command interface from port: 57052 20110905-000108 804 cmd1853 - Host:/127.0.0.1 closed command interface from port: 54365 20110905-000113 804 cmd1854 - Host:/127.0.0.1 closed command interface from port: 57052 20110905-000313 761 cmd1858 - Command: 20110905-000313 766 cmd1856 - Response: 201 Service ready 20110905-000313 766 cmd1858 - Response: 201 Service ready
I want to sort it in numeric order on the cmd value. Note that the cmd number isn't always 4 digits and I want cmd9 to appear before cmd10, etc.
Can I do this with sort?
I've seen examples that suggest that something along the lines of sort -n -k3.4 .. would do it but I can't get it working in any of the various permutations I've tried.
-- Mark Rogers // More Solutions Ltd (Peterborough Office) // 0844 251 1450 Registered in England (0456 0902) 21 Drakes Mews, Milton Keynes, MK8 0ER
sort -n -k 3.4,3.7
does it -- see man sort and the definition of -k POS1[,POS2] and the explanation of POS at the bottom of the man page.
I think your "sort -n -k3.4" only sorts on the first digit!
Ted.
-------------------------------------------------------------------- E-Mail: (Ted Harding) ted.harding@wlandres.net Fax-to-email: +44 (0)870 094 0861 Date: 06-Sep-11 Time: 13:58:37 ------------------------------ XFMail ------------------------------
OOPS! Sorry, I didn't read your query carefully enough and proceeded on the basis that the "cmd" numers were 4-figure. You will need to extract what follows "cmd" as a separate field. The following mix of awk, sed and sort works (the source file is called "temp.txt" here, and I have added a 3-figure "cmd" entry):
$cat temp.txt 20110905-000107 803 cmd1853 - Host:/127.0.0.1 opened command interface from port: 54365 20110905-000108 761 cmd1854 - Command: 20110905-000108 766 cmd1853 - Response: 201 Service ready 20110905-000108 766 cmd1854 - Response: 201 Service ready 20110905-000108 803 cmd1854 - Host:/127.0.0.1 opened command interface from port: 57052 20110905-000108 804 cmd1853 - Host:/127.0.0.1 closed command interface from port: 54365 20110905-000113 804 cmd1854 - Host:/127.0.0.1 closed command interface from port: 57052 20110905-000313 761 cmd1858 - Command: 20110905-000313 761 cmd858 - Command: 20110905-000313 766 cmd1856 - Response: 201 Service ready 20110905-000313 766 cmd1858 - Response: 201 Service ready
cat temp.txt | awk '{print $3 " " $0}' | sed 's/^cmd//' | sort -n | awk '{$1=""};{print $0}' | sed 's/^ //'
20110905-000313 761 cmd858 - Command: 20110905-000107 803 cmd1853 - Host:/127.0.0.1 opened command interface from port: 54365 20110905-000108 766 cmd1853 - Response: 201 Service ready 20110905-000108 804 cmd1853 - Host:/127.0.0.1 closed command interface from port: 54365 20110905-000108 761 cmd1854 - Command: 20110905-000108 766 cmd1854 - Response: 201 Service ready 20110905-000108 803 cmd1854 - Host:/127.0.0.1 opened command interface from port: 57052 20110905-000113 804 cmd1854 - Host:/127.0.0.1 closed command interface from port: 57052 20110905-000313 766 cmd1856 - Response: 201 Service ready 20110905-000313 761 cmd1858 - Command: 20110905-000313 766 cmd1858 - Response: 201 Service ready
Sorry for the previous error! Ted.
On 06-Sep-11 12:58:40, Ted Harding wrote:
On 06-Sep-11 12:35:57, Mark Rogers wrote:
I have a log file which looks like this: 20110905-000107 803 cmd1853 - Host:/127.0.0.1 opened command interface from port: 54365 20110905-000108 761 cmd1854 - Command: 20110905-000108 766 cmd1853 - Response: 201 Service ready 20110905-000108 766 cmd1854 - Response: 201 Service ready 20110905-000108 803 cmd1854 - Host:/127.0.0.1 opened command interface from port: 57052 20110905-000108 804 cmd1853 - Host:/127.0.0.1 closed command interface from port: 54365 20110905-000113 804 cmd1854 - Host:/127.0.0.1 closed command interface from port: 57052 20110905-000313 761 cmd1858 - Command: 20110905-000313 766 cmd1856 - Response: 201 Service ready 20110905-000313 766 cmd1858 - Response: 201 Service ready
I want to sort it in numeric order on the cmd value. Note that the cmd number isn't always 4 digits and I want cmd9 to appear before cmd10, etc.
Can I do this with sort?
I've seen examples that suggest that something along the lines of sort -n -k3.4 .. would do it but I can't get it working in any of the various permutations I've tried.
-- Mark Rogers // More Solutions Ltd (Peterborough Office) // 0844 251 1450 Registered in England (0456 0902) 21 Drakes Mews, Milton Keynes, MK8 0ER
sort -n -k 3.4,3.7
does it -- see man sort and the definition of -k POS1[,POS2] and the explanation of POS at the bottom of the man page.
I think your "sort -n -k3.4" only sorts on the first digit!
Ted.
E-Mail: (Ted Harding) ted.harding@wlandres.net Fax-to-email: +44 (0)870 094 0861 Date: 06-Sep-11 Time: 13:58:37 ------------------------------ XFMail ------------------------------
main@lists.alug.org.uk http://www.alug.org.uk/ http://lists.alug.org.uk/mailman/listinfo/main Unsubscribe? See message headers or the web site above!
-------------------------------------------------------------------- E-Mail: (Ted Harding) ted.harding@wlandres.net Fax-to-email: +44 (0)870 094 0861 Date: 06-Sep-11 Time: 14:14:28 ------------------------------ XFMail ------------------------------
On 06/09/11 14:14, (Ted Harding) wrote:
OOPS! Sorry, I didn't read your query carefully enough and proceeded on the basis that the "cmd" numers were 4-figure. You will need to extract what follows "cmd" as a separate field. The following mix of awk, sed and sort works (the source file is called "temp.txt" here, and I have added a 3-figure "cmd" entry):
I was concerned that it might be messy like that! (The actual files I'm processing are around 100MB in size so messy means very messy.)
I can't see why the -n or -g switches shouldn't be able to apply but I can't make them work!
Thanks for trying though, I'll play with the awk/sed route and see whether I can make that work for me. For my purposes it won't really matter if I just strip the "cmd" and use sort with -nk3, I just didn't think it should be necessary.
this appears to work on my setup. sort --key=3.4 -b -n < log
produced this (from a slightly modified logfile). 20110905-000313 761 cmd9 - Command: 20110905-000113 804 cmd10 - Host:/127.0.0.1 closed command interface from port: 57052 20110905-000108 766 cmd185 - Response: 201 Service ready 20110905-000107 803 cmd1853 - Host:/127.0.0.1 opened command interface from port: 54365 20110905-000108 766 cmd1853 - Response: 201 Service ready 20110905-000108 804 cmd1853 - Host:/127.0.0.1 closed command interface from port: 54365 20110905-000108 761 cmd1854 - Command: 20110905-000108 803 cmx1854 - Host:/127.0.0.1 opened command interface from port: 57052 20110905-000313 766 cmd1856 - Response: 201 Service ready 20110905-000313 766 cmd1858 - Response: 201 Service ready
On 06-Sep-11 16:06:56, Mark Rogers wrote:
On 06/09/11 16:54, nev young wrote:
sort --key=3.4 -b -n
Perfect, thanks!
-- Mark Rogers // More Solutions Ltd (Peterborough Office) // 0844 251 1450
Interesting! The key (pun intended) to the problem lies in the separator. Nev's "-b" (accpording to 'man sort') has the effect that:
-b, --ignore-leading-blanks ignore leading blanks
and the separator option:
-t, --field-separator=SEP use SEP instead of non-blank to blank transition
So, in your typical input line:
20110905-000108 761 cmd1854 - Command:
the default separator occurs at the places indicated below by "|":
20110905-000108| 761| cmd1854| -| Command:
and the blanks after each "|" are part of the following field. so that the 3rd field is " cmd1854". Hence the start of the numeric part is at position 5, not position 4. The leading " " in each field is eliminated by "-b".
Test (using my test file with an extra 3-digit line):
sort --key=3.5 -n < temp.txt ` 20110905-000313 761 cmd858 - Command: 20110905-000107 803 cmd1853 - Host:/127.0.0.1 opened command interface from port: 54365 20110905-000108 766 cmd1853 - Response: 201 Service ready 20110905-000108 804 cmd1853 - Host:/127.0.0.1 closed command interface from port: 54365 20110905-000108 761 cmd1854 - Command: 20110905-000108 766 cmd1854 - Response: 201 Service ready 20110905-000108 803 cmd1854 - Host:/127.0.0.1 opened command interface from port: 57052 20110905-000113 804 cmd1854 - Host:/127.0.0.1 closed command interface from port: 57052 20110905-000313 766 cmd1856 - Response: 201 Service ready 20110905-000313 761 cmd1858 - Command: 20110905-000313 766 cmd1858 - Response: 201 Service ready
which is perfect, and also explains why your "-k3.4" did not work!
Well well well!
Another way to do it, of course, would be
sort -t " " --key=3.4 -n < temp.txt
Ted.
-------------------------------------------------------------------- E-Mail: (Ted Harding) ted.harding@wlandres.net Fax-to-email: +44 (0)870 094 0861 Date: 06-Sep-11 Time: 18:58:37 ------------------------------ XFMail ------------------------------
On 06/09/11 18:58, (Ted Harding) wrote:
Interesting! The key (pun intended) to the problem lies in the separator. [..snip..]
Interesting indeed, I'm pleased you worked it out (and more so that you then went on to explain it).
I played with this for far too long before posting here, and one thing that I couldn't work out was why the character counts didn't seem to work as I expected them to. Suddenly it all becomes clear!