Arch Linux

mpz

Member

Registered: 2010-10-14

Posts: 54

File Sync

I work on a system where transfer of files between servers occurs daily in following order:

Client > Production > Staging > Dev

We use lftp to mirror Client and Production servers (since we only have SFTP access) and rsync to propagate files internally. What I don't like is that we have to eyeball cronjobs for Staging and Dev servers. Since I don't know how long it'll take lftp to mirror Client with Production, I have to give it ample time before I can call rsync on Staging. Same issue with Staging and Dev. Ideally, I would have Production trigger sync job on Staging once it got data from the Client and Staging trigger sync job on Dev boxes once it got data from Production.

Other then implementing some flavor of network file system, are there any other solutions out there to help us achieve this?

Thanks

fukawi2

Ex-Administratorino

From: .vic.au

Registered: 2007-09-28

Posts: 6,237

Website

Re: File Sync

Wrap the rsync in a script that waits until a file disappears before starting rsync:

this_server='production'
next_server='staging'
sync_file='sync_in_progress'

# Function to touch a file on a remote server (exercise for the reader)
create_file_on_remote_server $next_server $sync_file

# wait until the sync *to* this server has completed
while [[ -e $sync_file ]] ; do
   sleep 5s
done

rsync -arg1 -arg2 --exclude=$sync_file source/ dest/

# Function to remove a file on a remote server (exercise for the reader)
remove_file_on_remote_server $next_server $sync_file

exit $?

You can then start these scripts within 1 minute or so of each other, and each sucessive server will wait until $sync_file is deleted before starting it's rsync

---

Are you familiar with our Forum Rules, and How To Ask Questions The Smart Way?

BlueHackers // fscanary // resticctl