Announcement

Collapse
No announcement yet.

Ignore Unimportant Differences doesn't seem to be working

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Ignore Unimportant Differences doesn't seem to be working

    I'm on BC x64 Windows, v4.3.7 build 25118.

    When I do ctrl-A "compare file contents", View>Ignore Unimportant Differences doesn't seem to affect anything. Also the "~~ Minor" button (which I think is the same function) has no effect.

    Even after pressing F5 to refresh. Even after doing ctrl-A then "compare file contents" again.

    See attached screenshots.

    Lots of the .lua files in the screenshots are identical except for timestamp (and maybe whitespace), yet they show as different in Compare Contents - no matter how I set Ignore Unimportant Differences.

    (If you want, I'm happy to post the source files I'm comparing - it's all open-source code that's online.)

    Advice?

    Click image for larger version

Name:	ss1.png
Views:	168
Size:	80.3 KB
ID:	85220Click image for larger version

Name:	ss2..png
Views:	100
Size:	35.5 KB
ID:	85222Click image for larger version

Name:	ss3.png
Views:	102
Size:	82.0 KB
ID:	85221

  • #2
    Hello,

    Are you running a Rules-based content scan? The Content Comparison (center column) can run as a CRC, Binary, or Rules-based scan. The 0110 help signify that the current results are binary/crc based (which, don't use unimportance). Sometimes, the rules-based scan still still show these results if it can short-circuit the results, which is why I ask what the configuration is, but it should be just "=" if the short-circuit continues into the configured rules-based results.

    You can double check in the Folder Compare's Session menu -> Session Settings, Comparison tab. Or if using the Compare Contents command, make sure the Tools menu -> Options dialog, File Operations, Confirmation, Confirm content compare, is enabled, so you can select rules-based.

    Double clicking on a file always runs a rules-based scan, and then overrides the previous results, as it's what you just "looked" at.
    Aaron P Scooter Software

    Comment


    • #3
      Originally posted by Aaron View Post
      Are you running a Rules-based content scan? The Content Comparison (center column) can run as a CRC, Binary, or Rules-based scan. The 0110 help signify that the current results are binary/crc based (which, don't use unimportance). Sometimes, the rules-based scan still still show these results if it can short-circuit the results, which is why I ask what the configuration is, but it should be just "=" if the short-circuit continues into the configured rules-based results.

      You can double check in the Folder Compare's Session menu -> Session Settings, Comparison tab.
      Forgive me, I'm confused by that answer. What's "0110 help"? All I'm doing is:

      1 - Select 2 folders in Windows
      2 - Right-click "Compare". (This starts BC.)

      At this stage I see the results of what I think you call the "quick comparison" - one that compares based on directory information (timestamps, file size, etc.) instead of opening and reading files. That's fine and it works as expected.

      But then:

      3 - Ctrl-A to select all files (in both folders)
      4 - Right-click, and choose "Compare contents"

      I expect that to read each file (on each side) and compare the contents. Not just compare filenames, file size, etc. (Of course it takes longer - or should.)

      But it doesn't. It seems to be doing a "Quick Compare". I thought perhaps the "~~ Minor" button had something to do with it, but it doesn't seem to affect the result.

      EDIT: Maybe it does open them all and read them, I'm not sure (the files are small so it's hard to tell).

      My problem is that the process above (steps 1 to 4 above) marks most files as different - which is also OK because* they are in fact binary different (the ones on the the left use Unix line endings (\n) the ones in the right use Windows line endings (CR LF). This also makes the file sizes slightly different.

      Other than that the files are identical. I expect the "~~ minor" button to ignore the "minor difference" of line endings (and whitespace differences), but I can't figure out how to get that comparison to run on the whole folder.

      The only way I'm able to confirm the files are identical (except for line endings and whitespace) is to double click each one in BC, then close it. One by one. That opens the comparison window, showing no difference. When I close that window the one file now shows as "=".

      As in my first post above:

      The first screenshot shows the two folders being compared, after having opened gui_icon.lua as described, then closed the window. Only that file is marked as "=". (The files are shaded because I'd selected all files with Ctrl-A before taking that screenshot).

      The second screenshot shows what I see after double-clicking exposure.lua - again, no differences visible (which is as it should be).

      The last screenshot above shows what I see after switching from the exposure.lua window ("pane"?) to the folder pane - now exposure.lua is marked as "=". Which is what I wanted to know in the first place - for all files, not just gui_icon.lua and exposure.lua.

      This is probably user error on my part, but can you please explain how to get what I want?


      I did check Session>SessionSettings>Comparison; see attached screenshot. Please advise.

      Click image for larger version  Name:	ss.png Views:	0 Size:	26.7 KB ID:	85281

      -----

      Added: While I have your attention, sometimes I compare two folders on different filesystems. The "Quick Compare" marks them as different until I open each file and compare contents. (In fact the files are binary identical.)

      It appears BC thinks they're different because the timestamps don't match exactly - either because of rounding differences in the filesystem (they're' a few seconds off) or because of timezone differences (one filesystem is reporting local time, the other is accounting for timezones).

      Frequently I work with files created in a different timezone than where I am now - when files get copied sometimes Windows corrects the the timestamps so they'll be "correct" in the place where the copied files go, sometimes it doesn't, depending on the age of the systems that created and copied (can be 20 years apart - for example files were created on in 1997 on Windows 95, then copied on Windows 7 in 2017, now compared on Windows 10 in 2021...).

      Is there a workaround so BC will report more accurately in those cases? I tried increasing the 2 second tolerance in the box above - that didn't seem to help.
      Last edited by Dave92F1; 24-Feb-2021, 08:36 AM.

      Comment


      • #4
        There's a lot here, but I think we want to rewind a bit. There are a couple of fundamental interface decisions in BC4 that you'll want to internalize to understand the comparison.

        First, file names align left to right, and the name gets a color, based on the defined comparison criteria (View menu -> Legend). However, the data on the screen can be different than what is compared. For example, you can disable timestamps as a comparison criteria, but still show the timestamp column.

        Central to the above screenshot is the center column between the files, with have a "=" or "0110=" icon. This is the content comparison results column. The "0110" prefix in front indicates if a CRC or Binary scan was used, and the lack of that binary prefix means a rules-based scan was used. The color determines the results (black for equal, and red for unequal).

        By default, BC4 runs a timestamp/size comparison, which updates the color of the file names but doesn't populate the center column. However, if you ever double click on a file pair and view the results, that is a rules-based scan, so we update the parent with those results (in the center column).

        The Session Settings dialog can be updated to run any of the three content scans. The manual Compare Contents command can also perform that scan on a selection. However, if you double click on the files, this still runs a Rules-based scan and then updates with those results. This means viewing the file can overwrite a previous binary/crc scan with newer/different Rules-based scan results.

        Binary/CRC are a bit by bit scan of the file, while Rules-based is like opening and comparing the visible data in the file viewer, which can ignore some changes by default. Such as whitespace or line ending characters (as you note). This means a file with different line endings will be binary different, but rules-based same.

        This is the role of the Minor (Ignore Unimportant Differences) toggle on the toolbar. In a Text view, you have the same toggle, and it will shift Unimportant Blue text to act as black/equal text. When disabled, red and blue are both Differences for the purposes of the comparison, while when enabled only Red changes remain and act as differences. In the level above that, the Folder Compare can treat Similar files (files that contain Only Unimportant Differences) as either Different or Same, depending on the toggle state. This way, the parent Folder Compare can hide files that contain only (defined) unimportant changes when viewing with the Show Differences display filter, or show them if you disable the toggle.

        In your Session Settings dialog screenshot, to get the results you expect, you should leave Timestamp, Size, and Override Quick tests enabled, then also enable Compare Contents: Rules-based comparison. This auto-runs (like double clicking every file) the rules-based results on load, and allows them over 'override' the timestamp difference, while still allowing timestamps to help determine if a file is Newer or Older when a rules-based change is detected.


        Does this help explain the interface and why double clicking the files is updating/altering the results in the center column, which then overrides the quick tests (timestamp/size) results?
        We have a KB article on the subject here, for reference: https://www.scootersoftware.com/supp...ferentthensame

        The same Comparison tab also has an option to ignore Timezone differences of exactly an hour, which can also include the Tolerance setting (if it needs to be 5 seconds or more, depending on your files). Another fix: if you run a rules-based scan, determine they are equal, you can then use the Touch command to copy the timestamp from one side to the other and resolve them to be equal. I would not recommend this yet, until you are more comfortable with the program and the results it provides, as a Touch (and other actions) cannot be Undone (there is no Undo command).
        https://www.scootersoftware.com/support.php?zz=kb_dst
        Aaron P Scooter Software

        Comment


        • #5
          Thanks; I really appreciate the attempt to explain.

          I use BC every day, and have for years. That doesn't mean I ever figured out everything it can do (or will ever, probably). So let me ask directly:

          1 - I'm going to keep the default at "quick compare" because it's fast. I normally start BC from File Explorer with "select left folder" and "Compare to".

          Given that:

          If I want a binary compare of all files, I know I can do ctrl-A, right-click, "Compare contents".

          If I want a rules-based compare of all files, what's the quickest way to do that? (I don't see "Rules-based compare" on the right-click menu, so it's not obvious to me.)

          2 - Re the '0111' icon - does the bit pattern mean anything (is it always 0111 or can it be for example 1010)?

          3 - When I see '0111', I think you're saying that indicates the result of 'compare contents'. If so, is that always a binary compare, or can it be a CRC compare? If it can be either, how to choose?

          4 - (Just because I've always wondered), what is the purpose of offering both a CRC and binary compare? For a sufficiently long CRC (I'd think 64 bits would be enough) they should always (for all practical values of "always") give the same result. (Esp. if you don't tell anybody what CRC polynomial you're using.)

          I don't think a CRC compare is any quicker (unless the OS pre-computes it and you can just look it up in the directory). If so, you have to read all of both files either way (CRC or binary), so why not just omit the CRC option and offer only a binary compare? (It would simplify things a little.)

          Thanks...

          Last edited by Dave92F1; 26-Feb-2021, 08:19 PM.

          Comment


          • #6
            Ah hah! I think I know the main issue (see #1)

            1) Go to the Tools menu -> Options dialog, File Operations and check "Confirm content compare". This will cause a dialog to pop up when you Ctrl+A + Right-click Compare Contents. The dialog allows selection Binary or Rules-based, and has a "don't show this again" option if you always want to use a specific one (which, if ever clicked, requires re-enabling the dialog in the global options).

            2) The 0111 icon does not mean anything as a pattern, it's a "binary" pattern to signify a binary scan (or crc scan) was run, and the number is static. It's a pictoral graphic.

            3) The 0111 icon can mean either binary or crc, and doesn't differentiate between them. They are considered equally valid as 'binary' type comparisons and aren't ignoring any data like a rules-based scan can. You'd have to know which you selected to do if the difference is meaningful to you. The choice is made in the Session Settings or in the Compare Contents dialog (once we resolve #1). One place it can crop up is if you have selected to run a rules-based, it can be faster to run a Binary scan first, and if BC4 runs that scan instead as a short-circuit and it returns equal then we return those results (since a binary equal file must also be rules-based equal).

            4) There are pros/cons to each scan. For example, the Binary scan goes bit-by-bit and as soon as a difference is encountered between the pair, it can stop computing and return different. The CRC must fully complete the scan to get the hash, and then use the hash for the comparison. Meanwhile, the CRC scan can work with an FTP server that supports the xCRC command, so temp files don't need to be downloaded if the server supports xCRC to compute them remotely. It also helps with archives where the CRC is pre-computed. The CRC we use is CRC-32, same as used by .zip and POSIX chksum.
            Aaron P Scooter Software

            Comment


            • #7
              Originally posted by Aaron View Post
              Ah hah! I think I know the main issue (see #1)

              1) Go to the Tools menu -> Options dialog, File Operations and check "Confirm content compare". This will cause a dialog to pop up when you Ctrl+A + Right-click Compare Contents. The dialog allows selection Binary or Rules-based, and has a "don't show this again" option if you always want to use a specific one (which, if ever clicked, requires re-enabling the dialog in the global options).
              That was EXACTLY what I needed - thanks!!

              Comment

              Working...
              X