Performance Question on Filtering on Unimportant Differences

Collapse
X
Collapse
 
  • Page of 1
  • Time
  • Show
Clear All
new posts
Previous template Next
  • nbd66m4
    Enthusiast
    • May 2011
    • 25
    #1

    Performance Question on Filtering on Unimportant Differences

    I have 192 XLSX files I converted to CSV files all about 45K in size.

    I have a process that compares same files twice, and posts results to website, results color coded. Once with filters, once without.

    My performance doesn't change whether all work is performed locally on a W2K8 based Windows server or if output files are posted to another server.

    I can run a non filtered file comparison in batch in less than 2 minutes creating HTML result file output.

    If in my script I choose to ignore differences, the session fill ignore 6 RegExp date formats and 3 string formats, the process in batch now runs over 3 hours for those same 192 45K CSV files.

    If I run it in the GUI as a folder compare, takes less minute to do the exact match versus fuzzy match. If I click each file the HTML is shown instantly, and I'm pretty sure if I clicked each of the 192 compares, I'd see HTML results for all in less than 5 minutes.

    I'm not sure what all I need to show for RegExp rules, script, a sample file?

    Maybe the issue is in my side, I just don't know. I know filtering stuff over each full line each line isn't free, but wow, 3 hours.

    Any help would be appreciated.

    doug
    Tags: None
    • Aaron
      Team Scooter
      • Oct 2007
      • 16000
      #2
      In the GUI, you can enable to run a Rules-based scan (Session menu -> Session Settings -> Comparison tab: leave timestamp, size, override enabled, also enable Rules-based) to "click" every file.
      http://www.scootersoftware.com/suppo...ferentthensame


      Assuming a Folder Compare, you could configure the session settings and filters to work as expected in the GUI, then in script:
      load "Session Name"
      to load a session with criteria, filters, etc already set.

      Would it be possible to post or email your current script file (before any other modifications/testing)? You can email us at [email protected], and please include a link back to this forum post for our reference.
      If you email us, you can also include your BCSupport.zip (Help menu -> Support; Export) which would include all of your settings, sessions, etc as well.
      Aaron P Scooter Software

        Comment

        • nbd66m4
          Enthusiast
          • May 2011
          • 25
          #3
          Sent compare details and attachments

          to [email protected] referencing this thread.

            Comment

            • Aaron
              Team Scooter
              • Oct 2007
              • 16000
              #4
              I had a few specific script questions I've emailed back, along with details on more specifically the steps taken in the graphical interface you wish to line up with.
              Aaron P Scooter Software

                Comment

                • nbd66m4
                  Enthusiast
                  • May 2011
                  • 25
                  #5
                  That did the trick

                  I'm not sure why I missed the difference in the 2 scripts (filtered, non-filtered). Not even understanding the fix, I should have at least noticed it.

                  Thank you, this did scale the filtered compare dow to similat non-filtered timings.

                  Much regards,

                  doug

                    Comment

                    Previous template Next
                    Working...