interrupted compares

[150d]

Hi,

I'm frequently using BeyondCompare to transfer a directory tree between two machines over a slow network link (VPN). Often the total amount of data is so big that a complete transfer takes many hours.

What happens if the connection breaks before the transfer is completed? More specifically, what happens to the file that had been in transfer at the time: Is the (incomplete) target version of the file deleted by BC or will it remain?

In other words: If a file is in the target tree after BC has been exited, can I assume that it has been completly transferred?

Regs,
150d

[Chris]

If you're concerned about a dropped connection during a transfer, you should probably run a comparison later with BC using the default criteria (timestamp and size) to make sure all of the files have completely transferred.

You can also consult BC's log, any errors during the transfer should be listed in the log.

[150d]

A comparison is not that easy - either I transfer all the data again to make a binary compare, which again will take hours and again may be interrupted, or I only do a quick compare as you suggest using size and timestamp and no content compare.

But can I assume that especially filesize is correct? It would be no problem to write two bytes data and label the file two MB in size on disk.

If I read you correctly, I can NOT assume that any not-completly-transfered file will also never show up in the target tree, e.g. will be deleted when BC finds it cannot complete the transfer? There IS the very real risk that such a half-file may persist?

Regs,
150d

[Chris]

The fastest way to handle this is to look at error messages in BC. If you're running BC interactively, it will display an error message if files failed to copy. If you're running a script, add the script command "log c:\logfile.txt" to the beginning of the script. If there is a failure to copy it will be listed in the log file.

[Zoë]

It's going to depend on how you're transferring the file (FTP, SFTP, network share), and what the remote server does. Some servers will transfer it to a temp location, and some won't. Since you said the connection dropped, there's no way for BC to delete the remote file, but even if it was a transient error, BC does not explicitly delete files if a transfer error occurs; we used to do that for FTP transfers and it caused more problems than it helped with.

The safest thing to do is test it. Start transferring a largish file and pull your network cable until the transfer times out.

[150d]

Thank you for your replys. Maybe I didn't give enough information about my situation, sorry about that:

I'm using two Windows machines. The remote machine is the source, the local machine is the target. The connection is established over VPN using a share of the remote machine. The transfer direction is remote-to-local. Data consists of about 500 files, about 4 GB in total. Transfer takes over a day.

When the connection breaks, BC detects this, aborts and prints error messages that the path could not (no longer) be found, how many files were transfered, and how many expected files could not be transfered. It e.g. states

"07.01.2010 05:42:03 Successfully copied 254 items. Failed to copy 322 items. Completed in 15 hours, 0 minutes."

Above that are 322 (I guess ;-) ) messages that a path could not be found. It doesn't say which file had been in transfer when the break occured.

The point is: Will there be exactly 254 items (are these just files or do directories count, too?) on the traget drice after BC ceases operation? Or might there be cases in which there are 255 on the local target drive, but the last one (having been in transfer when the connection broke) is only partially completed?

I would like to have a clear distinction: Either the file was transfered completely, then it will show up in the target tree. If it was only partially completed because an error interfered, then it will be deleted. Of course this can only be the case for a local target (as opposed to a remote target.)

Does BC already act this way?

Regs,
150d

[Chris]

Hi 150d,

We don't explicitly delete a file if the connection drops partway through the copy. We use a Windows file copy function (CopyFileEx) to copy the files, so the behavior should match Windows Explorer.

In my own tests it looks like a partial file shouldn't be left on your local disk if the connection drops, but I can't guarantee it.

[150d]

Thank you for the clarification.

Regs,
150d

[150d]

One more thought: If a half finished file should remain, what file size will it have? The "intended" one or the size of data that had in fact been transfered?

[Michael Bulgrien]

If a half finished file should remain, what file size will it have? The "intended" one or the size of data that had in fact been transfered?

In my past experience with Windows, the size of the target file is often set before a file copy completes. In these cases, a failed copy is not detectable by file size, but will be evident if you check the file's CRC.