No announcement yet.

BC4 Crashing, Hanging, Slow & other General Frustrations

  • Filter
  • Time
  • Show
Clear All
new posts

  • BC4 Crashing, Hanging, Slow & other General Frustrations

    I've been trying to use BC (Version 4.1.9 (build 21719)) to make sure that all data has been copied accurately between two HDDs before I re-format the first.

    The Finder isn't very good at large copy operations, and I don't necessarily trust it to do these accurately (with good reason). I have often used BC in the past to verify copies (on both Windows and Mac), and it has usually worked OK (albeit slower than I'd like).

    I love BC4 when it works (which it usually does for smaller operations), but I've been trying for over a week now to copy & verify about 3TB (a million files or so) from one HDD to another, and BC always either hangs or crashes before completing the operation.

    I have tried:
    • Using BC to do the copy in Folder Compare mode
    • Using BC to do the copy in Folder Sync mode
    • Using the Finder to do the copy, and using BC to do a Binary comparison of the contents
    • Using the Finder to do the copy, and using BC just to compare the directory trees (not even content)

    It's never managed to get all the way through ANY of these tasks. It sometimes manages to get through smaller chunks OK, but trying to let it run through larger chunks overnight has always resulted in disappointment the next morning. If I'm not greeted with a crash dump, I'm greeted with a spinning multi-colored wheel (aka pizza) of death. Very frustrating.

    Other frustrations:
    1. Most of the time, when I try to terminate one of these larger binary comparison sessions (by clicking the flashing red "X" Stop button), BC becomes (remains?) unresponsive. Often the red "X" continues to blink. I generally need to force-quit BC4 to get it to work again.

    2. There is no real progress indicator for the overall comparison session, so it's often very hard to tell how far BC has gotten, or if it is still making any progress at all. I often have to resort to Activity Monitor to see if it's still reading data or making much use of the CPUs. Often (but not always) Activity Monitor will report that BC is "not responding". I wish there was an overall progress bar, like there is when I copy/move files in BC.

    3. While it is still actively working, BC uses a lot of HD bandwidth (as expected). This can make other operations on the machine very slow (as expected). I wish there was a way to PAUSE & RESUME a long comparison session, similar to how you can pause and resume copy, move and delete operations (although that's also been kind of hit or miss, in my experience).

    4. When BC crashes (or I have to force-quit it), most of the times I find upon re-launch that my session was NOT auto-saved, and so I have to start over again from scratch. I'm slowly learning to manually save each session before starting it, but what is the point of auto-save if it doesn't? It seems that auto-save should automatically save (and flush to disk) every session immediately upon it being started.

    5. I can't quite understand why it should take BC hours just to compare two directory trees (no content comparison). It seems that it should be able to read both directory trees entirely into RAM and blast through them very quickly (even with a million nodes). But it seems to take a MUCH slower, piecemeal, approach.

    6. For binary comparisons, I have till recently used the default buffer size (which is a seemingly-paltry 64K). I've recently tried bumping that up to several GB, but it seems that the maximum BC allows is 999999999 bytes. I would suggest that the default be set considerably larger than 64K, and that users be allowed to specify buffer sizes in units of MB (or at least KB), rather than in individual Bytes! With modern systems often having 32+ GB RAM, and file sizes in a similar range, it would be nice if the upper limit were raised as well.

    I am running a fresh install of OS X 10.12.1 off an SSD in a 2010 Mac Pro w/ 32GB of RAM (which passes extensive testing). Comparisons are between two internal SATA HDDs, both of which check out just fine with Disk First Aid. BC4 seems to leak memory, but is still nowhere close to running out of available RAM.

    Many of these issues have been around a long time (some on Windows, too), but I usually am able to muddle through them. But this particular time has been more challenging and frustrating. I probably can get through it, if I break the comparisons down into smaller pieces, but it's hugely frustrating to have to do so, manually keeping track of which directories and subdirectories have been verified, and which ones have not. And not being able to just set BC loose overnight means I have to let it run in the background while I'm trying to get work done, which is also very frustrating.

  • #2

    BC4 for Mac is a 32bit application, and would still be under those memory limitations even if your system has more RAM accessible. When we crash, we would generally show our crash dialog which can present an OutOfMemory error (although not always, if that crashes, too). Do you see our crash dialog, the system crash dialog, or a silent failure/close? Reading the entire directory structure and storing all of that information is intensive and generally the memory limit we hit if dealing with millions of files; the binary scan itself isn't the intensive, memory using part of the process. Most other software only loads the currently visible tree and file structure, while BC4 must load all subitems and metadata in order to generate the comparison.

    We may well also have a bug or memory leak that you are hitting that we'll need to troubleshoot, but I just wanted to clarify why your 32GB of system memory is not being utilized.

    If you do get the crash dump from the system, emailing that in to with a link back to this forum thread would be useful for determining what is going on.

    The binary buffer controls how much of a buffer we use at once per binary scan, but this doesn't need to be a large number to increase performance. In fact, if you are triggering multiple binary scans at once manually, this would quickly eat up the available ram we need to load the directory structure and we'd run out soon. I'd suggest turning this number back down to 32megs or 64megs at most if BC4 is running into other memory issues loading the full comparison.

    And a couple of the other, related issues you are hitting while running into this:
    - BC4's Abort command waits for a response from the OS; if this is hung or requires a timeout (most often seen with slow NAS devices) we have to 'wait to hear back'.
    - A foreground selection -> right-click -> Compare Contents: Binary scan can be paused.
    - We auto-save on close, so crashes can interrupt this. I would recommend setting up the session, then going Home and back (or manually Save Session) as needed.
    Aaron P Scooter Software