Diff'ing is much slower in BC4 than BC3

[svec]

BC Version 4.0 beta (build 17432)

I diff two text files in BC3 and it takes 4.47 seconds.

BC4 takes 2 minutes, 4 seconds.

The files are too big to upload, so here's a way to generate them (run in cygwin on Win7):

dd if=/dev/urandom bs=64k count=1 > random-bytes.bin
xxd random-bytes.bin > a
dd if=/dev/urandom bs=64k count=1 > random-bytes.bin
xxd random-bytes.bin > b

[Zoë]

In the Session settings dialog on the "Alignment" tab change the "Skew tolerance" setting from 4000 to 2000. We increased the size for BC4 and that will greatly affect the speed of the comparison if there are a lot of differences.

Why are you running the files through xxd though? Beyond Compare's hex compare is designed for binary files and will do a better job of comparing them than your current approach with the text compare will.

[svec]

Thanks for the suggestion: I changed the skew to 2000 and now it only takes 60 seconds, but that's still more than 10x slower than BC3 on the same files.

I set the skew to 500 and the load time is down to 15.1 seconds - better, but still 4x slower than BC3.

15 seconds isn't too bad, but I regularly diff larger files than this, and those load times will be much worse.

I don't have the binary files, I'm diff'ing the text files because that's the only output I have.

The "xxd" flow was a way to tell you what the files I'm diffing look like, it's not that actual flow I'm using - I couldn't upload my actual files because they were too big.

[Zoë]

Ok, that sounds better. We do have some sample files that are significantly slower in BC4 than BC3. I haven't had a chance to profile things to figure out where the slowdown was introduced, but we will have it fixed before the final release. A quick test shows that whatever it is only affects the "Standard" alignment though; if you change to either the "Myers O(ND)" one (formerly "alternate") or the new "Patience diff" one the speed should be much better.

Also, if your files were originally generated using xxd, it does have a -r option that reverses the operation, producing the original binary files. You can even set up a file format to run it automatically before comparing them in the hex compare. That would still be my recommendation to get the best comparison results.

[Zoë]

I tracked the major slowdown down; the next release should be similar in speed to BC3.

[svec]

The recent beta is MUCH better - thank you VERY much! :-)