Using RSYNC with Amazon S3 [closed]

I recently stumbled across this thread on Google and it looks like the landscape has changed a bit since the question was asked. Most of the solutions suggested here are either no longer maintained or have turned commercial.

After some frustrations working with FUSE and some of the other solutions out there, I decided to write my own command-line rsync “clone” for S3 and Google Storage using Python.

You can check out the project on GitHub: http://github.com/seedifferently/boto_rsync

Another project which I was recently made aware of is “duplicity.” It looks a little more elaborate and it can be found here: http://duplicity.nongnu.org/

Hope this helps.

UPDATE

The Python team at AWS has been working hard on a boto-based CLI project for their cloud services. Among the tools included is an interface for S3 which duplicates (and in many ways supersedes) most of the functionality provided by boto-rsync:

https://github.com/aws/aws-cli

In particular, the sync command can be configured to function almost exactly like rsync:

http://docs.aws.amazon.com/cli/latest/reference/s3/sync.html

Leave a Comment