I've been trying for over an hour to concatenate a load of .vcf files for import into another device.
I need to have a carriage return and line feed between each vcf so I created a spacerfile which contained just those two characters.
I then generated a list of all the files with the ls command. I then used that file like this :- while read file ; do cat $file spacerfile.txt >> all_in_one.vcf ; done <listfile.txt
But all I get is a load of blank lines with just one or two vcards inserted and then a long list of the vcf files which is most of the listfile.txt.
I am also seeing this type of error :- cat: First: No such file or directory cat: Name.vcf: No such file or directory
That made me wonder if it was because there are spaces in the file names and hence in the listfile.txt so I amended the text file so that each space was preceded by the escape character. But it still doesn't work.
Where am I going wrong?
On Tue, Feb 18, 2014 at 06:11:34PM +0000, Chris Walker wrote:
I've been trying for over an hour to concatenate a load of .vcf files for import into another device.
I need to have a carriage return and line feed between each vcf so I created a spacerfile which contained just those two characters.
I then generated a list of all the files with the ls command. I then used that file like this :- while read file ; do cat $file spacerfile.txt >> all_in_one.vcf ; done <listfile.txt
But all I get is a load of blank lines with just one or two vcards inserted and then a long list of the vcf files which is most of the listfile.txt.
I am also seeing this type of error :- cat: First: No such file or directory cat: Name.vcf: No such file or directory
That made me wonder if it was because there are spaces in the file names and hence in the listfile.txt so I amended the text file so that each space was preceded by the escape character. But it still doesn't work.
From the error messages you're seeing I think "because there are
spaces in the file names" is exactly your problem.
I don't think I'd do it quite the way you have, spaces in filenames in a shell script are always very painful to handle.
I think I'd do something like the following, on the command line with no scripts:-
for fn in *.vcf do cat "$fn" >>listfile echo >>listfile echo >>listfile done
You can type it just as I have listed it above as the shell will prompt for the intermediate lines with a '>' until you enter the line with 'done'.
On Tue, 18 Feb 2014 18:28:43 +0000 Chris Green cl@isbd.net wrote:
On Tue, Feb 18, 2014 at 06:11:34PM +0000, Chris Walker wrote:
I've been trying for over an hour to concatenate a load of .vcf files for import into another device.
I need to have a carriage return and line feed between each vcf so I created a spacerfile which contained just those two characters.
I then generated a list of all the files with the ls command. I then used that file like this :- while read file ; do cat $file spacerfile.txt >> all_in_one.vcf ; done <listfile.txt
But all I get is a load of blank lines with just one or two vcards inserted and then a long list of the vcf files which is most of the listfile.txt.
I am also seeing this type of error :- cat: First: No such file or directory cat: Name.vcf: No such file or directory
That made me wonder if it was because there are spaces in the file names and hence in the listfile.txt so I amended the text file so that each space was preceded by the escape character. But it still doesn't work.
From the error messages you're seeing I think "because there are spaces in the file names" is exactly your problem.
I don't think I'd do it quite the way you have, spaces in filenames in a shell script are always very painful to handle.
I think I'd do something like the following, on the command line with no scripts:-
for fn in *.vcf do cat "$fn" >>listfile echo >>listfile echo >>listfile doneYou can type it just as I have listed it above as the shell will prompt for the intermediate lines with a '>' until you enter the line with 'done'.
OK. I've followed your instructions and I do indeed end up with a listfile comprising all the vcf entries.
But I don't understand why it works so can you explain it for me please? The problem I have is why am I running the echo line twice?
Ta.
On Tue, Feb 18, 2014 at 06:38:42PM +0000, Chris Walker wrote:
On Tue, 18 Feb 2014 18:28:43 +0000 Chris Green cl@isbd.net wrote:
I think I'd do something like the following, on the command line with no scripts:-
for fn in *.vcf do cat "$fn" >>listfile echo >>listfile echo >>listfile doneYou can type it just as I have listed it above as the shell will prompt for the intermediate lines with a '>' until you enter the line with 'done'.
OK. I've followed your instructions and I do indeed end up with a listfile comprising all the vcf entries.
But I don't understand why it works so can you explain it for me please? The problem I have is why am I running the echo line twice?
The 'for' executes everything between the 'do' and the 'done' for each .vcf file. The 'cat' does what you know, I put quotes round the file name to make it work with filenames that have embedded spaces. 'echo' just outputs what's after it, which in the above case is nothing except for CR/LF (end of line). I just did two echoes to give you two blank lines.
On Tue, 18 Feb 2014 20:39:00 +0000 Chris Green cl@isbd.net wrote:
On Tue, Feb 18, 2014 at 06:38:42PM +0000, Chris Walker wrote:
[snip]
But I don't understand why it works so can you explain it for me please? The problem I have is why am I running the echo line twice?
The 'for' executes everything between the 'do' and the 'done' for each .vcf file. The 'cat' does what you know, I put quotes round the file name to make it work with filenames that have embedded spaces. 'echo' just outputs what's after it, which in the above case is nothing except for CR/LF (end of line). I just did two echoes to give you two blank lines.
Thanks for the explanation. It's much appreciated.
On 18 Feb 20:39, Chris Green wrote:
On Tue, Feb 18, 2014 at 06:38:42PM +0000, Chris Walker wrote:
On Tue, 18 Feb 2014 18:28:43 +0000 Chris Green cl@isbd.net wrote:
I think I'd do something like the following, on the command line with no scripts:-
for fn in *.vcf do cat "$fn" >>listfile echo >>listfile echo >>listfile doneYou can type it just as I have listed it above as the shell will prompt for the intermediate lines with a '>' until you enter the line with 'done'.
OK. I've followed your instructions and I do indeed end up with a listfile comprising all the vcf entries.
But I don't understand why it works so can you explain it for me please? The problem I have is why am I running the echo line twice?
The 'for' executes everything between the 'do' and the 'done' for each .vcf file. The 'cat' does what you know, I put quotes round the file name to make it work with filenames that have embedded spaces. 'echo' just outputs what's after it, which in the above case is nothing except for CR/LF (end of line). I just did two echoes to give you two blank lines.
ENOTTRUE, echo, by default, won't do CR/LF, just LF. About the only place that a CR/LF pair is still used is in DOS format text files.
If you'd wanted a CR/LF pair, you could use:
echo -ne '\r\n\r\n'
Which would be 2 new lines in DOS text file format.
If you just wanted 2 new lines, you could just use:
echo -ne '\n\n'
Also, the for loop has the issue that if *.vcf expands too much (i.e. if there's lots and lots of files in the list), then it'll also fail.
So, really, it should have gone something like...
find . -maxdepth 1 -name *.vcf -print0 | while read -d $'\0' -r file; do (cat "$file" && echo -ne '\n\n') >> all.out; done
Which deals with interesting filenames, only uses 1 redirection, and should be marginally quicker.
Thanks,
ENOTTRUE, echo, by default, won't do CR/LF, just LF. About the only place that a CR/LF pair is still used is in DOS format text files.
It depends how you interpret things.
It certainly *does* echo a CR/LF pair to the screen because otherwise the cursor wouldn't return to the LHS.
I do agree though that it will only write an LF to the file. :-)
My answer did what the OP wanted anyway! :-)
On 19 Feb 16:44, Chris Green wrote:
ENOTTRUE, echo, by default, won't do CR/LF, just LF. About the only place that a CR/LF pair is still used is in DOS format text files.
It depends how you interpret things.
It certainly *does* echo a CR/LF pair to the screen because otherwise the cursor wouldn't return to the LHS.
EWRONGAGIN ;)
To prove this:
echo -ne "test\n"
I.e. *only* echo the line feed, no carriage return. Note that your terminal still returns the cursor possition. See for example:
$ echo -ne "test\ntest2\n" test test2 $
(try it yourself if you don't believe me :)
UNIX systems have only ever bothered using the line feed character to mean both, it's up to the terminal what to do.
I do agree though that it will only write an LF to the file. :-)
My answer did what the OP wanted anyway! :-)
That's true :)
On Wed, Feb 19, 2014 at 04:51:36PM +0000, Brett Parker wrote:
On 19 Feb 16:44, Chris Green wrote:
ENOTTRUE, echo, by default, won't do CR/LF, just LF. About the only place that a CR/LF pair is still used is in DOS format text files.
It depends how you interpret things.
It certainly *does* echo a CR/LF pair to the screen because otherwise the cursor wouldn't return to the LHS.
EWRONGAGIN ;)
No I'm not, we're arguing at cross purposes.
echo -ne "test\n"I.e. *only* echo the line feed, no carriage return. Note that your terminal still returns the cursor possition. See for example:
Only because something in between is adding the CR. If the terminal was in a different mode where the CR isn't added the cursor wouldn't go to the LHS.
UNIX systems have only ever bothered using the line feed character to mean both, it's up to the terminal what to do.
Yes, absolutely, a much more sensible approach than CR and LF. But it only works because of processing in between by the terminal driver which adds the CR back when outputting to real-world devices.
-
Brett Parker -
Chris Green -
Chris Walker