Out of Memory error during large folder compare

[davenovak]

I am doing a rather large folder sync across the LAN and I'm getting a crash after the session loads and shortly into syncing. The folder I'm syncing is about 225GB and contains close to 1.5 million files (yes, I know that's a lot). Though the error logs contain a variety of errors, I've seen "out of memory" pop up quite a bit.

I realize that this must be pushing the boundaries of BC3, but this same sync operation worked under BC2.

Any suggestions?

[davenovak]

UPDATE --

After breaking the sync up into smaller pieces, I managed to get the entire folder synced (only about 30% of the files had changed). Now, with the folders up-to-date, I've seen that smaller changes (in the neighborhood of 10,000 files) can be synced up automatically using the root sync (which failed previously). Therefore, the problem seems to be not so much the overall size of the folder but the number of items that need to be synced.

Final comment -- I used a Mirror to left operation to sync these folders (in case that maters).

[Zoë]

Hi Dave,

Thanks for the update. Any idea how many files needed to be synched when it was failing? Also, are you synching over FTP or via shared folders?

[davenovak]

That's really hard to know how many files were being synced, though my guess would be something less than .5 million. Would the crash logs contain that information? I have those if they would shed any light on the issue. Also, I emailed those logs through the (very cool) BC3 error reporting feature.

This was not FTP. I was syncing using shared folders over the LAN.

[Aaron]

It is possible the sync you are attempting is beyond BC3's limits.
http://www.scootersoftware.com/suppo..._maxfilev3.php

Try cutting it up into smaller pieces (syncing subfolders, using file filters, etc). Does this work for you?