Not exactly a Linux question, but the answer may equally apply and it's confusing the living daylights out of me so-
I have a client with a SCO box...the application it runs is sometimes made to generate a file that has to magically appear a bit later on another (Windows) box.
Due to visionFS (sort of like Samba but not as nice) not being completely happy talking to Windows Server 2003 (or vice versa) the method that was chosen when this was implemented by whoever was FTP.
So there is a cron job that looks roughly like this
#!/bin/sh userid=foo pword=bar host=windowsbox file=transferfile.csv
if [ -e $file ] then ftp -i -n $host <<putfile user $userid $pword cd /location/on/windows/box lcd /file/location put $file putfile echo file transferred mv $file $file.sent else exit
The odd thing is that the file is nearly always truncated...I thought at first we were getting unlucky and the script was transferring it before app had finished writing it. So I tried inserting a wait in the script just before the ftp transfer bit...So then it looked for the file and if it existed waited 2 minutes (plenty of time for the app to finish with the file) and only then transferred it...If anything this seemed to make things worse!!!
Also interestingly the $file.sent (ie the moved copy of $file) is always complete. But I am guessing because it's a move it would be because the unlinked filename handle would be working until the app let go of it right ?
oh and it's not the ftp transfer that's causing the problem because I can do the transfer manually and all is well. Can't help thinking that there is a better way of doing this...but any ideas on how to re-implement or fix the current method are very welcome.
Hi Wayne
On Thursday 01 June 2006 23:06, Wayne Stallwood wrote:
The odd thing is that the file is nearly always truncated...I thought at first we were getting unlucky and the script was transferring it before app had finished writing it. So I tried inserting a wait in the script just before the ftp transfer bit...
Does forcing the ftp transfer to use binary mode help at all ?
Regards, Paul.
On 01-Jun-06 Wayne Stallwood wrote:
Not exactly a Linux question, but the answer may equally apply and it's confusing the living daylights out of me so-
I have a client with a SCO box...the application it runs is sometimes made to generate a file that has to magically appear a bit later on another (Windows) box.
Due to visionFS (sort of like Samba but not as nice) not being completely happy talking to Windows Server 2003 (or vice versa) the method that was chosen when this was implemented by whoever was FTP.
So there is a cron job that looks roughly like this
#!/bin/sh userid=foo pword=bar host=windowsbox file=transferfile.csv
if [ -e $file ] then ftp -i -n $host <<putfile user $userid $pword cd /location/on/windows/box lcd /file/location put $file putfile echo file transferred mv $file $file.sent else exit
The odd thing is that the file is nearly always truncated...I thought at first we were getting unlucky and the script was transferring it before app had finished writing it. So I tried inserting a wait in the script just before the ftp transfer bit...So then it looked for the file and if it existed waited 2 minutes (plenty of time for the app to finish with the file) and only then transferred it...If anything this seemed to make things worse!!!
Also interestingly the $file.sent (ie the moved copy of $file) is always complete. But I am guessing because it's a move it would be because the unlinked filename handle would be working until the app let go of it right ?
oh and it's not the ftp transfer that's causing the problem because I can do the transfer manually and all is well. Can't help thinking that there is a better way of doing this...but any ideas on how to re-implement or fix the current method are very welcome.
I can't diagnose your script, Wayne (since it's SCO sh which I'm not familiar with; though if it were bash I'd be expecting some semicolons, perhaps, and certainly a "fi" to close the "if").
However, you might consider a diagnostic test on the lines of
#!/bin/sh userid=foo pword=bar host=windowsbox file=transferfile.csv
if [ -e $file ] then cp $file $file.copy #### <<<<<<<<<<<<<<<<<<< ftp -i -n $host <<putfile user $userid $pword cd /location/on/windows/box lcd /file/location put $file putfile echo file transferred mv $file $file.sent else exit
and then compare $file.copy a) with the file transferfile.csv which was supposed to be sent, b) with the file which is actually received.
This might help to focus on where the problem lies!
Good luck, Ted.
-------------------------------------------------------------------- E-Mail: (Ted Harding) Ted.Harding@nessie.mcc.ac.uk Fax-to-email: +44 (0)870 094 0861 Date: 01-Jun-06 Time: 23:17:20 ------------------------------ XFMail ------------------------------
On Thu, Jun 01, 2006 at 11:06:27PM +0100, Wayne Stallwood wrote:
#!/bin/sh userid=foo pword=bar host=windowsbox file=transferfile.csv
if [ -e $file ] then ftp -i -n $host <<putfile user $userid $pword cd /location/on/windows/box lcd /file/location put $file putfile echo file transferred mv $file $file.sent else exit
^ missing a fi, but that's probably not the issue ;)
The odd thing is that the file is nearly always truncated...I thought at first we were getting unlucky and the script was transferring it before app had finished writing it. So I tried inserting a wait in the script just before the ftp transfer bit...So then it looked for the file and if it existed waited 2 minutes (plenty of time for the app to finish with the file) and only then transferred it...If anything this seemed to make things worse!!!
Rough wondering guess... does this file *happen* to be a binary file? Could it, perhaps, contain a null character, or something else that wouldn't fit in with an ascii file transfer? Might be worth adding a command in there, the binary command, just before the put $file.
Also interestingly the $file.sent (ie the moved copy of $file) is always complete. But I am guessing because it's a move it would be because the unlinked filename handle would be working until the app let go of it right ?
That it would, you *could* wait on an lsof of the file to return nothing, I suppose... not sure if that even exists on that OS though, good luck!
oh and it's not the ftp transfer that's causing the problem because I can do the transfer manually and all is well.
Hmm, using exactly that set of commands? does it work if you run that script exactly as is?
Can't help thinking that there is a better way of doing this...but any ideas on how to re-implement or fix the current method are very welcome.
Hmm, maybe use smbclient to talk to the windows box? Or maybe ditch the windows box entirely, you don't really need it, right? ;)
Or the windows box could use NFS to talk to the SCO box... or maybe a different FTP client, something along the lines of lftp?
I hope you've got a large field full of Goat's that are ready for sacrificing ;)
Cheers,
You can get more info from the SCO side by adding -d to the ftp command line in /etc/inetd.conf then sending a kill -1 to inetd, this will log all ftp activity to /usr/adm/syslog. Don't forget to undo that at some point as it can add a significant amount of text to syslog.
I don't think that SCO support VisionFS anymore but they do have a downloadable for Samba if you need it. I did some Samba training at SCO from one of the Samba team was apparently employed by SCO.
Regards, Rob.
On 1 Jun 2006, at 23:06, Wayne Stallwood wrote:
Not exactly a Linux question, but the answer may equally apply and it's confusing the living daylights out of me so-
I have a client with a SCO box...the application it runs is sometimes made to generate a file that has to magically appear a bit later on another (Windows) box.
Due to visionFS (sort of like Samba but not as nice) not being completely happy talking to Windows Server 2003 (or vice versa) the method that was chosen when this was implemented by whoever was FTP.
On Thu, Jun 01, 2006 at 11:26:00PM +0100, Robert Tillyard wrote:
You can get more info from the SCO side by adding -d to the ftp command line in /etc/inetd.conf then sending a kill -1 to inetd, this will log all ftp activity to /usr/adm/syslog. Don't forget to undo that at some point as it can add a significant amount of text to syslog.
Surely that'd only log things for the SCO ftp daemon, he's talking to a windows FTP daemon, so surely this wouldn't help ;)
I don't think that SCO support VisionFS anymore but they do have a downloadable for Samba if you need it. I did some Samba training at SCO from one of the Samba team was apparently employed by SCO.
I didn't think SCO supported computing anymore, I thought they just started random court cases to accuse everyone else in the world of "stealing" their work. But that's a different matter entirely.
Cheers,
On 1 Jun 2006, at 23:43, Brett Parker wrote:
On Thu, Jun 01, 2006 at 11:26:00PM +0100, Robert Tillyard wrote:
You can get more info from the SCO side by adding -d to the ftp command line in /etc/inetd.conf then sending a kill -1 to inetd, this will log all ftp activity to /usr/adm/syslog. Don't forget to undo that at some point as it can add a significant amount of text to syslog.
Surely that'd only log things for the SCO ftp daemon, he's talking to a windows FTP daemon, so surely this wouldn't help ;)
Okay, sorry, my fault, I thought the transfer was the other way around. If it was it would have logged if the connection was shutdown or the transfer was complete.
As it is to the Windows box and as mentioned by others I can't see a line to set the transfer mode to binary, Windows does default to ASCII and the first ^Z will make Windows think the EOF marker has passed as well as causing CR/LF translation.
I don't think that SCO support VisionFS anymore but they do have a downloadable for Samba if you need it. I did some Samba training at SCO from one of the Samba team was apparently employed by SCO.
I didn't think SCO supported computing anymore, I thought they just started random court cases to accuse everyone else in the world of "stealing" their work. But that's a different matter entirely.
Not my area of expertise, I just develop for 300 SCO systems. I did attend a briefing once where the legal stuff was mentioned but it bored me to tears and I switched off.
Cheers,
Brett Parker
Regards, Rob.
On 1 Jun 2006, at 23:06, Wayne Stallwood wrote:
Not exactly a Linux question, but the answer may equally apply and it's confusing the living daylights out of me so-
I have a client with a SCO box...the application it runs is sometimes made to generate a file that has to magically appear a bit later on another (Windows) box.
Due to visionFS (sort of like Samba but not as nice) not being completely happy talking to Windows Server 2003 (or vice versa) the method that was chosen when this was implemented by whoever was FTP.
So there is a cron job that looks roughly like this
#!/bin/sh userid=foo pword=bar host=windowsbox file=transferfile.csv
if [ -e $file ] then ftp -i -n $host <<putfile user $userid $pword cd /location/on/windows/box lcd /file/location put $file putfile echo file transferred mv $file $file.sent else exit
The odd thing is that the file is nearly always truncated...I thought at first we were getting unlucky and the script was transferring it before app had finished writing it. So I tried inserting a wait in the script just before the ftp transfer bit...So then it looked for the file and if it existed waited 2 minutes (plenty of time for the app to finish with the file) and only then transferred it...If anything this seemed to make things worse!!!
Also interestingly the $file.sent (ie the moved copy of $file) is always complete. But I am guessing because it's a move it would be because the unlinked filename handle would be working until the app let go of it right ?
oh and it's not the ftp transfer that's causing the problem because I can do the transfer manually and all is well. Can't help thinking that there is a better way of doing this...but any ideas on how to re-implement or fix the current method are very welcome.
Howdy, I recently ditched a similar script in favor of curl. I had all manner of problems (script-wise and transfer wise) scripting ftp.
-Mark
Cheers for all the suggestions guys...I hadn't considered bin mode because it's 'supposed' to be a tab delimited plain text file...but there is a possibility there I suppose.
I think the only time I have tried running the script manually it has been with a dummy file (that I generated in vi) and the manual transfer that works with the proper file was done interactively from the ftp prompt (when I usually instinctively go to bin mode anyway)
curl looks interesting though
sadly it's going to have to wait until tomorrow now...I don't have access to the box tonight to play
-
Brett Parker -
Mark Ridley -
Paul -
Robert Tillyard -
Ted.Harding＠nessie.mcc.ac.uk -
Wayne Stallwood