Binary vs. CRC issue

[Tom]

This is weird -- I just did a folder comparison of 100 megs of files each in build 456. CRCs normally take a fraction of the time to calculate that it takes to compare an entire file, byte by byte.

The CRC comparison report:
5/26/2008 2:41:09 am Successfully compared 55 items. Completed in 7.75 seconds.

The binary comparison report:
5/26/2008 2:41:17 am Successfully compared 55 items. Completed in 1.34 seconds.

Looks like sumpin's wrong somewhere...

[Zoë]

CRCs and binary comparisons should generally take about the same amount of time to perform. In this case the CRC calculations made the operating system cache much of the files' contents in memory, so the binary comparisons didn't actually have to hit the disk. If you don't want that to happen you'll need to use the "Bypass disk cache during binary comparisons" checkbox in the Session Settings dialog.

[Tom]

CRCs and binary comparisons should generally take about the same amount of time to perform.

I didn't realize that! In BC2 they always took much longer. Thanks.

I must've misunderstood the purpose of CRC comparisons, then -- I thought it was a less-detailed comparison that sampled the file mathematically. What's the benefit of using CRCs?

[Zoë]

The only time where there's CRCs are really advantageous over binary comparisons is when BC can get the CRCs without reading the files contents and computing the CRCs itself. For example, Zip files store CRCs as part of the file headers, so it can use those without extracting the files, and if an FTP server supports the XCRC command it can compute the CRC on the remote side and just send that instead of transferring the entire file.

If the files you're comparing are on the same disk CRC comparisons can be faster than binary comparisons just because it's able to read the files sequentially instead of alternating between them, which can take advantage of operating system level caching. In BC3 there's a tweak (Ctrl+Shift+T), the name of which eludes me, that increases the size of the buffers it uses for binary comparisons. You can increase that value to get similar speed to CRC comparisons in those cases. In BC2 it's the BinaryBufferSize tweak, documented at http://scootersoftware.com/support.php?c=kb_tweaks.php

[Tom]

In BC3 there's a tweak (Ctrl+Shift+T), the name of which eludes me, that increases the size of the buffers it uses for binary comparisons.

Great idea -- thanks! (It's "Buffer size for binary compare" and defaults to 65536 bytes.)