On 06/06/13 18:26, Srdjan Todorovic wrote:
Hi,
On 6 June 2013 16:32, Mark Rogers mark@quarella.co.uk wrote:
I need to download a set of backup files from a (Windows) server, deleting all but the newest from the server once successfully downloaded.
Something like Expect [0] could really help you here. You'd have to write all the file age, already-have-a-copy-of-a-file (etc) logic yourself though.
Wow - Expect looks interesting! I'd not seen that before.
Mark, I can think of several possibilities, all of which involve some sort of automation - shell script or batch file. These would require checking the status of a command to see if it suceeded: e.g. Linux: Not entirely sure how, but perhaps something along the lines of what's described here see http://stackoverflow.com/questions/3822621/how-to-exit-if-a-command-failed
Windows Batch File: Somecommand if errorlevel==1 goto SomeCommandFailed
With the windows batch file, you must check that "Somecommand" actually sets the error level if the command fails. I think from memory, that means success, and >0 means failure. That example, I think would goto SomeCommandFailed for any return value greater than 0 - but it's been a while - check!
I'm assuming you don't know in advance the names of the files you're copying.
You could a) use expect as Srdjan suggested
b) Install cygwin on the windows server. Use rsync to copy all the relevant files from Windows to your server. Rsync will only copy files that need copying, omitting files that are the same on both sides. (alternatively use scp) If controlling this from your server's side, use ssh to send commands to the windows server to delete the old/copied files. If controlling this from the Windows server, use DOS command to delete the files you've already copied - not entirely sure how to calculate the date of the files in as Windows Batch file. Alternatively, you could use a vbscript program on Windows - this would give you more flexibility in calcuating the dates.
c) Windows Centric: Launch the process from the windows machine. Set up a scheduled task to launch a batch file, vbs program or some other program to do stuff for you. If you try to do a mass ftp transfer off all files in one go (I'm not sure that you can), then if it fails, you'll probably have to re-transfer the whole lot as you won't know which file failed. So, you'll probably have to copy one file at a time.
A very very simple set of batch files to do this would look something like
MainFile.BAT CD MyDirectory
|FOR %%A IN (/*.BAK/) DO CALL C:\BATCHDirectory\MYFTP %%/A /IF ERRORLEVEL == 1 GOTO FAILED GOTO SUCCESS
:FAILED Echo "Failure" END 1
:SUCCESS Echo "Success" END 0
||MYFTP|.BAT IF "%1"=="" GOTO NOPARAMS
FTP %1 REMOTESERVERPATH 'or whatever the format of the FTP command is... IF ERRORLEVEL == 1 GOTO COPYFAILED ECHO "Successfully copied %1" END 0
:COPYFAILED ECHO "Failed to copy copied %1" END 1
:NOPARAMS ECHO "Must specify a file name" END 2
You could do the same in vbscript or some higher level program, but then you'd have to work out how to call a dos command from vbs (it is do-able). You then need to calculate which files to delete and delete them - this would be much easier in a vbscript program. I *Think* you can calculate dates of files in a batch file, but off the top of my head, I can't remember how!
d) Windows centric Just throwing this out there. Investigate if there are winscp or wsftp can do the copying for you and be scripted adequately.
e) Linux Centric Install the FTPCopy (debian/ubuntu) package Use ftpls to generate a listing of the windows server's files. Use ftpcopy to copy the files from the remote server (copying whole directories is supported) Compare the remote file list with the current directory and see if all files have transferred, if yes, work out which ones to delete. Use ftp to delete the remote files, probably one at a time
f) linux cetnric Use straight ftp command with get, mget or reget to fetch remote files. If they transfer OK, use mdelete to delete the ones on the remote machine.
Hope that helps in some way!
I think where you go depends on your familiarity with the various environments, e.g. do you do Batch files or vbscript, or are you more comfortable with linux scripts or perl, php etc?
My inclination would be to do the cygwin way. Then a) you're using linux at both ends b) you have the linux tools you're familiar with at both ends c) you can use rsync to efficiently transfer the files (do it over ssh if you need them to be secure) d) you can issue a command locally something like this to delete the old files on the windows server
|ssh user@WindowsServer|find /path/to/files* -mtime +2 -exec rm {} ;
See: http://www.howtogeek.com/howto/ubuntu/delete-files-older-than-x-days-on-linu... http://stackoverflow.com/questions/305035/how-to-use-ssh-to-run-shell-script...
That's enough for now! Good luck!
Cheers Steve