2 Amazon S3 issues...

[ron_hiner]

I was a long time fan of BC on Windows. I move to mac a few years ago, and I'm thrilled to see that BC is now available for the mac! Yay! Even more thrilled that BC supports Amazon S3!

I'm trying to sync a large collection of photos (maybe a 2-3 TB) from my local storage to Amazon S3. I've run into two problems in my testing....

1) When I copy files from my mac to my S3 drive, all works fine. The file timestamps on the S3 files remain the same as the local files. But if I "Synchronize" then the resulting file date/time on S3 records the current time as the file date/time, rather than the original. Therefore, since the file timestamps are different, they are out of sync with the local files. Why would Synchronize treat file timestamps than would Copy?

2) Many of my image files are somewhat big... like 40 mb. After a little photoshop, these files can easily get to be 10 times that size. BC seems to error out on a dropped connection long before the file is transferred, so perhaps the timeout value is set to short to make big file transfers possible. Any solution to this? (According to speedtest.net, my upload speed is 30 mb/s.)

Thanks in advance!

running BC 4.0.7 on Mac OSX 10.10.4

[Aaron]

Hello,

Thanks for the report. Currently, Amazon S3 doesn't support setting the Last Modified timestamp, so our expectation would be that it would always get the timestamp of the time of transfer. This would be for Copy or Sync. I tested and reproduced this behavior on my own test machine.

You can update the local file's timestamp using the Touch command, as a means to help set the timestamps to be equal, but to set the remote side Amazon would need to allow this behavior.

Thanks for the report on large files. We'll look into what might be causing this to happen.

[ron_hiner]

Currently, Amazon S3 doesn't support setting the Last Modified timestamp, so .....

That seems like a deal breaker for using BC and S3 as a means to do offsite back up and manage recency with the last modified timestamp.

Are there other cloud storage options that you can recommend?

Thanks

[Aaron]

Currently, the Cloud storage profiles we support either do not support setting the timestamp or the API we use to communicate directly with them does not support it. It's something we would like to work with, but can be limited by the exposed API. I would suggest contacting Amazon and letting them know this is a feature you would like; they may log your request or be aware of a good workaround. We can be included on any of those emails: [email protected] with a link back to this forum thread for our reference.

For preserving dates, you can use a service which exposes itself as an FTP(SFTP/FTPS), WebDAV, or remote drive letter. If using a local folder to sync/store (rather than our Profile to directly connect), the native clients for Dropbox or OneDrive support keeping the Last Modified date.

[Chris]

Large file transfers to Amazon S3 are fixed in Beyond Compare 4.1.3.

[Aaron]

Behavior tweaked further to help with more cases in BC 4.1.4. If you were still having trouble, give this version a try and let us know any issues.

[tfrost]

Regarding the inability to update S3 modification times: what we have done to work around this (not with Beyond Compare), is to add a metadata item whenever our application uploads a file, showing the actual time of the file. Then in the separate application which allows an end user to download an installer from our S3 repository, we can show the actual file time rather than the upload time, which may up to 24 hours later.

It might be useful for people like the orignal poster, if Beyond Compare had an option to do something similar on upload of files to S3, and could then use the metadata item, if found, when displaying comparisons. But it does take longer to download the metadata when retrieving a list of bucket contents, so there is a cost to this convenience.