No announcement yet.

Proper Sync Script without Proper Datestamps

  • Filter
  • Time
  • Show
Clear All
new posts

  • Proper Sync Script without Proper Datestamps

    I normally use the SYNC MIRROR script commands to back up files internally from my current server locations to a backup NAS, which works wonderfully. The commands are simply:
    Load <sessionname>
    option confirm:yes-to-all
    sync mirror: lt->rt
    I have another NAS box at a remote location, to which I send data periodically as a backup from my own systems, but this is NOT working quite so well. Apparently, this particular NAS FTP server doesn't support setting the timestamps of the files via FTP. Because of this, ALL of the files on the target server have datestamps that are from the last backup -- even if they did not change on my source location.

    Of course, because I use the SYNC MIRROR command, BC sees the newer dates, assumes that the files are wrong, and forces EVERY file to be copied, which increases the time (and bandwidth) for backup by quite a bit. I don't want to use UPDATE, because my in-house copy is the master, and I just want to mirror this, but I cannot use MIRROR, either, because of the timestamps.

    After digging further, I think this should be able to be accomplished with a more complicated script. However, I am unsure of the EXACT script that could simulate this. I am thinking along these lines:
    Load <SessionName>
    Option confirm:yes-to-all
    Select right.orphan
    Delete right
    Select left.orphan
    Copy left->right
    Select left.newer
    Copy left->right
    I know that the files on the right side will NEVER be newer, because this is a backup target only. Will this accomplish what I want? Are there any easier ways to do this?

  • #2
    Here's a simpler script that does the same thing as the script you wrote:

    load <session name>
    select right.orphan
    delete right
    sync update:left->right

    The above script will delete files that only exist on the right. It will also copy files that only exist on the left and files with newer modified timestamps on the left to the right.

    SFTP usually supports preserving timestamps on copy. If your NAS supports SFTP it might be a good workaround to the timestamp issue.
    Chris K Scooter Software


    • #3
      Thanks for the suggestion and simplification. I will indeed try this.

      The Buffalo Terastation supports an SFTP connection, but it does not fare any better than the FTP connection. I have the BC option "Preserve Timestamps on Upload" checked, but this doesn't help. I did a manual test where I delete an existing file from the right side, then copy the file over from the left to the right. It gets timestamped with today's date and time every time.

      If I do this on a regular FTP session, I get a log which tells me the operation is invalid:
      2/28/2017 4:17:35 PM Sent> PASV
      2/28/2017 4:17:35 PM Recv> 227 Entering Passive Mode (50,79,57,235,164,72).
      2/28/2017 4:17:35 PM Sent> STOR
      2/28/2017 4:17:35 PM Recv> 150 Opening BINARY mode data connection for
      2/28/2017 4:17:35 PM Recv> 226 Transfer complete
      2/28/2017 4:17:35 PM Sent> MFMT 20151013220942
      2/28/2017 4:17:35 PM Recv> 501 Operation not permitted

      However, if I do this under the SFTP connection, I do NOT get a useful log -- I only get one line:
      2/28/2017 4:18:55 PM Successfully copied 1 items. Completed in 2.52 seconds.
      But the timestamp never gets set. I chalk this up to a problem with the Terastation firmware, and not a problem with BC, because I use the same thing to transfer data to several other NAS devices and even remote servers via FTP, and they all work just fine.


      • #4

        Thanks for that info. SFTP does not provide as much logging information as FTP, but we'll display whatever we receive back from the server.
        If you test transfer with another SFTP client, like Filezilla, do you see the same timestamp behavior (set to time of transfer)? It's a common issue for some servers to not support setting the timestamp, so the files receive the time of transfer instead.
        Aaron P Scooter Software