Folder sync over FTP issue...

[DanSwitzer2]

I maintain a website that has thousands of files. I've found the only reliable way to sync the files is to do a CRC comparison check--which makes syncing extremely slow.

Idealistically what would be perfect is if BC3 stored a cache of the CRC on the remote server and would only update the CRC check if the time stamp/file size on the remote server has changed.

It seems to do something similar to this when a session is left open. So as long as I keep BC3 open, it's not that bad syncing. But leaving the app open isn't ideal.

Is there a way in BC3 to save the state of the remote session? Just adding some comparison to the old remote folder's CRC state would drastically speed up performance for me.

Any ideas of what I might do to speed things up?

Thanks,
Dan

[Chris]

Hi Dan,

If you have control of the FTP server software running on the server, using a different FTP server might help. Some FTP servers can generate CRC values on the server, it usually speeds up CRC comparison pretty dramatically. The name of the FTP command is "XCRC", so you'll need to find a server with support for the command.

BC doesn't have a way to cache the CRC values. If you're the only one updating the web site, you could save a snapshot of your local files after every sync. The next time you do a sync, compare the local files to the snapshot, then use the "Copy to Folder" command to copy the changed files to the FTP server.

[DanSwitzer2]

Hi Dan,

If you have control of the FTP server software running on the server, using a different FTP server might help. Some FTP servers can generate CRC values on the server, it usually speeds up CRC comparison pretty dramatically. The name of the FTP command is "XCRC", so you'll need to find a server with support for the command.

BC doesn't have a way to cache the CRC values. If you're the only one updating the web site, you could save a snapshot of your local files after every sync. The next time you do a sync, compare the local files to the snapshot, then use the "Copy to Folder" command to copy the changed files to the FTP server.

Is there any chance that caching a CRC check is something that could be implemented as a feature?

I would think if you cached the remote file's time stamp and it's CRC, you could then use the cached CRC to do a compare as long as the time stamp hasn't changed--otherwise you calculate the new CRC.

The FTP server we're using doesn't support XCRC, and from my (admittedly short) research it doesn't appear to be a standard--although there are several FTP servers that support it.

[Chris]

Hi Dan,

I discussed it with our developers, caching a CRC check is a planned feature for BC 3.2.

[Chris]

Our developers just clarified this for me. "Caching" a CRC value isn't a feature that will be in the folder compare. For BC 3.2, we're working on "sync memory" to remember the state of the Folder Sync. If you use the "Folder Sync" in BC 3.2 (when it is released) to sync your files it might help out in your situation.

[DanSwitzer2]

Excellent!

Now of course the next question is when is 3.2 planned for?

Thanks!

[Aaron]

We do not have a set eta for 3.2. It is something we are currently working on, along with any bug fixes needed in the minor releases (3.1.x).