"Compare Contents" Rules-based comparison extremely slow

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • vletroye
    Visitor
    • May 2011
    • 9

    "Compare Contents" Rules-based comparison extremely slow

    Hi,

    I am comparing the content of a folder on my NAS with the content of a folder on my File Server.
    The File Server is simply mirroring the NAS. I am using a rules-based comparison to check that all files are complete on both side.

    When I select two files and start the comparison, Beyond Compare announces about 10' of duration.
    If I stop the comparison (after one minute, a few seconds or even immediately) and restart it, the result is immediate !

    I can do that several times for various files and I am able to reproduce the problem each time.

    I am using Beyond Compare 4.1.9 (trial). But I was able to reproduce the same issue with an old Beyond Compare 3.

    I am using UNC paths to compare the content of the Nas and the File Server. I have a Gigabit Ethernet and the bandwith is > 50MB/s in one direction. The NAS is a Synology (DSM 6.0.2). The File Server is a Windows 10 with SnapRaid + StableBit DrivePool. I.e.: all files on the File Server are in a pool!!

    I wanted to do a video illustrating my issue: http://sendvid.com/wyfkyssq
    Unfortunately, Windows 10's video recorder didn't capture the popup windows. So you won't see that I am correctly selecting the "rules-based" comparison. You won't see either the Version of BC and the Rules (they are the defaults). If I find another free recorder, I can post a complete video. But still, on can see that I start comparisons, that they don't progress, that I stop them, restart them, and it's then immediate.
  • Aaron
    Team Scooter
    • Oct 2007
    • 15941

    #2
    Hello,

    If you repeat the test using NAS <-> Local Desktop Test File, then Local Desktop Test File <-> Win10 SnapRaid, does one of these configurations fail? This would help us pinpoint if the issue is specific to one of the devices or needs both of them.

    When working with remote locations, files are downloaded locally as temp files to work with the Rules-based comparison scans. It sounds like the initial request to get these files is transferring slowly or not at all, while re-requesting or refreshing is causing the download request to succeed.

    The video link appears to have some intrusive ads. If you are having trouble capturing video, I'd recommend trying another software solution, like Techsmith's Jing:
    https://www.techsmith.com/jing.html
    Last edited by Aaron; 09-Jan-2017, 11:20 AM. Reason: Video issues
    Aaron P Scooter Software

    Comment

    • vletroye
      Visitor
      • May 2011
      • 9

      #3
      I am really surprise that files are downloaded (you mean "copied") locally for rules-based comparisons. My Files are from 6 to 10GB (ISO/HD movies/...).

      I am using a rules-based comparison precisely because it only compares file information such as timestamp and file size




      But indeed, if it's copying locally each file, it will be slooooow..
      But I am using Shared Folders, so I don't see why it would download the files.And also, if I stop immediately the comparison and restart it, the files can't for sure not be downloaded... and I have however a result.

      I did compare my NAS (HADES) with a local version of a file...
      09-01-17 18:57:14 Load comparison: \\HADES\movies\Movies HD <-> C:\temp

      A duration of 3 minutes were announced. So I stopped it after 11 sec:
      09-01-17 19:00:22 Failed to compare 1 items. Cancelled by user. Completed in 11 seconds.

      I did restart it and again, it was immediate
      09-01-17 19:00:27 Successfully compared 1 items. Completed in 0,00 seconds.

      I did compare next my snapraid (ZEUS) with a local version of file.
      A duration of 4 minutes were announced. So I stopped it and restart it. Again, it was immediate.

      09-01-17 19:02:21 Load comparison: \\ZEUS\Storage\Movies\Movies HD <-> C:\temp
      09-01-17 19:02:23 Fast refresh
      09-01-17 19:02:45 Failed to compare 1 items. Cancelled by user. Completed in 5,61 seconds.
      09-01-17 19:02:53 Successfully compared 1 items. Completed in 0,00 seconds.

      Finally, I did compare the two local versions and surprise... it was slow tooo ! Again, I did stop it and restart the comparison. And I got the result immediately here too:

      09-01-17 19:04:47 Load comparison: C:\temp\NVIDIA <-> C:\temp
      09-01-17 19:04:57 Fast refresh
      09-01-17 19:05:06 Failed to compare 1 items. Cancelled by user. Completed in 3,65 seconds.
      09-01-17 19:06:18 Successfully compared 1 items. Completed in 0,00 seconds.

      Weird, isn't ?!

      Comment

      • Aaron
        Team Scooter
        • Oct 2007
        • 15941

        #4
        How are you "Restarting it"? A Refresh, Full Refresh, selection -> Compare Contents command? Is the center column populating correctly between the files you expect?

        "I am using a rules-based comparison precisely because it only compares file information such as timestamp and file size" is not a correct assumption. The Timestamp and Size criteria is controlled above in the Quick Test Results. A Rules-based compare is similar to the CRC or Binary scans, which are content scans. Rules-based is similar to double clicking and viewing the file in the default file viewer for that file type. These can optionally ignore certain text or other rules defined by the file viewer. The Rules-based scan performs this scan, then returns these results to the center column in the Folder Compare, which can override or join with the other criteria that is checked/enabled.

        Disabling the content scan in the Session Settings, and only using the quick tests (timestamp/size, which is the default) does not download files.
        Aaron P Scooter Software

        Comment

        • vletroye
          Visitor
          • May 2011
          • 9

          #5
          Ok, thanks a lot for clarifying the comparison modes!

          I was "restarting" by simply selecting indeed the contextual menu "=? Compare Content" on the two items for which I just stopped the running comparison.

          I can't understand how it's so fast for the second comparison... expect if there is some kind of cache where the files were partially compared during the previous attempt and if this aborted attempt is considered as "complete" by the second attempt ?

          I just tried again and it's still the same (My installed version is 4.1.9 build 21719.)

          within Beyond Compare, I select one file XYZ of 8 GB in my C:\temp\ and compare it with a copy located in C:\temp\COPY\ (My C drive is a SSD Samsung 850 Pro 256GB)

          At that time, they are both displayed in black (meaning that they appear identical based on initial Load Comparison, right ?)
          I click on the menu "?= Compare Contents...", pick the entry "Rules-Based comparison".
          The progress bar appears.. it's sloooooow (>40 sec)
          I click on the with cross in a red disk next to the progress bar to stop it
          I click again on the menu "?= Compare Contents..." and pick the entry "Rules-Based comparison".
          The progress bar does not appear (well, I don't "see" it).. and a symbol like "0111 =" appears in the middle column.

          To reproduce the same behavior again on those files, I have to do a fresh (CTRL-F5).

          NB:
          - I don't have that behavior with a CRC comparison. it's always the same duration even if I stop it once and redo it.
          - a "Binary Comparison" is faster than a "Rules-base comparison"
          - a "Rules-base comparison" is also immediate if I did stop a "Binary Comparison" just before
          - a "Rules-base comparison" is not immediate if I did stop a "CRC Comparison" just before

          Comment

          • vletroye
            Visitor
            • May 2011
            • 9

            #6
            Here is a new video where one can now see me clicking in the various popups: http://sendvid.com/j6gpyrpt

            Comment

            • Aaron
              Team Scooter
              • Oct 2007
              • 15941

              #7
              Thanks, that looks like a bug we'll need to address. I'll open a tracker to investigate.

              For future videos, would it be possible to use a different service, such as youtube or a dropbox link? That website's ads are a security risk.
              Aaron P Scooter Software

              Comment

              • Chris
                Team Scooter
                • Oct 2007
                • 5533

                #8
                This bug is fixed in Beyond Compare 4.2 beta.

                Changelog:
                Fixed canceling a rules-based or binary comparison of a large pair of files caching incorrect CRC values and causing subsequent CRC or rules-based comparisons to report immediately matching contents.

                Beta site: http://www.scootersoftware.com/beta
                Chris K Scooter Software

                Comment

                Working...