No announcement yet.

Large Folder compare across WAN

  • Filter
  • Time
  • Show
Clear All
new posts

  • Large Folder compare across WAN

    Hi All,
    Firstly, congrats on some awesome software.

    My Problem
    I need to compare files on our central DFSR server which has 3.6 million files on it and replicates with several smaller sites.
    The largest of these small sites has around 1million files on it. I've tried to run this across the WAN (only a 2Mbps link) and it takes hours and hours. Is there any way to improve performance?
    My alternative is to use VB scripts to generate a list of files (.txt) at each location then compare the txt files with the central server for differences, this is really messy and will need about 50 scripts writing :/

    Any suggestions much appreciated!

  • #2

    Would using our BC Snapshots (in the Tools menu) to generate snapshot files be sufficient for your comparison purposes? You can then transfer the snapshot file and open it instead of the actual location. The snapshot acts as a virtual representation of the directory, and can contain the folder paths, file names, timestamps, size, and even CRC.
    Aaron P Scooter Software


    • #3
      Ahhhhhhh, that's one feature I hadn't spotted !
      So at first glance I think that will work. All I need to do now is....

      Get a license for each remote site so I can run it locally.
      Setup a way of scripting all this so that a local snapshot file is created at each local site (possible?)
      Copy the snapshot files to the central host
      Setup more scripting to compare the snapshots with the data on the central host
      Send a difference report to my email address

      Would you say all of the above is possible?

      thanks for your help so far


      • #4
        Yes, I believe so.

        If it is just you using the software, you would only need one license to install on those multiple machines. Please see our Per-User License section for more details:

        BC Scripting does have a snapshot command that can be used to generate the snapshot. You can find reference material in our Help file, in the Using Beyond Compare -> Automating with Scripts section, or in the Scripting Reference section.

        Scripting is a series of sequential steps, so you can perform a step to create a snapshot, then compare, or have multiple, specialized script files that you then call from a bat file (one after the other) as part of a larger automated script. We don't have a script command to send an email, so the last step BC3 would be able to perform is to generate the difference report (at which point, another script or program could email it).

        I would suggest performing a couple of the comparisons manually using the graphical interface first. This way you can test if snapshots meet your needs, and which report type and layout you would like to use. Scripting mirrors the actions you take in the graphical interface, so once you have a good workflow worked out, it will be easier to script the solution.
        Aaron P Scooter Software


        • #5
          Thanks for your assistance. I'm now able to compare our satellite sites with our main site each night, create output by email and html to display on a neat webpage for admins to view in the morning.
          Now, if only you had an agent that BC could deploy remotely, just when it needs to do a scan on a remote system, that returns the bcss file, that'd be perfect!



          • #6

            Your license allows you to install BC3 on multiple computers as long as you are the only user running the application. Are these remote machines constant? You could install your copy of BC3 to each to generate your snapshots.
            Aaron P Scooter Software


            • #7
              Yes, I have installed BC at the remote sites. I was just commenting that it would be even easier if BC could do a high speed local folder compare on any remote server without having to install BC first. The only way would be by deploying some sort of temporary agent while it ran the query. It's not a big deal though.
              My only remaining issue now is that I don't have access to certain folders/files so I can't get an accurate comparison report on one of our servers. Not a fault with BC though.