Announcement

Collapse
No announcement yet.

Mirror with versioning?

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Mirror with versioning?

    I have been using BC2 successfully in my digital photography workflow, but there is one feature that I am desperately trying to find a solution to... one that I hope I can achieve with BC2:

    Every day I have a mirror of my photo collection from one drive to another. The files are under subdirectories of year and then year-month-day.

    I need the ability to have it mirror "left to right", but with the following enhancements:
    <ul type="square">[*]If a file is new on the left (and doesn't exist on the right), copy it over with the same directory hierarchy[*]If a file is newer on the left (and exists on the right), then create a subdirectory on the right with today's date and then copy over the changed file to this subdir[/list]

    I am really hoping that there might be some way to achieve this with BC2, as it appears to be a relatively uncommon feature that has huge potential.

    By doing this, one achieves something much more robust than simple mirroring. With basic mirroring, you may ruin a file / photo and not realize it until after the mirror job, at which point you now have lost your original. With this enhancement, you always have the option of rolling back your changes if you need to.

    Thanks,
    Cal.

  • #2
    Re: Mirror with versioning?

    Thank you for the suggestion. I'll add it to the wish list.

    There are two ways you can get close to what you're describing with BC2.

    1) Use BC's Backup to Folder Option.

    In the Backups section of Tools|Options, check "Backup files before copy" and select to save backups to a folder. This doesn't provide an option to save multiple backups, and it isn't possible to specify a date wildcard in the folder name.

    2) Use a BC script.

    It is possible to expand the %date% environment variable in a BC script to create your backup directory. The default for %date% on U.S. English systems in XP is to format the date with / characters, which aren't valid for file names. To change to a different date format, open the Control Panel. Open "Date, Time, Language, and Regional Options". Open "Regional and Language Options". Click Customize in the Regional Options tab. Go to the Date tab. Change the Date Separator under "Short date". I changed mine to the - character.

    And here's the script to use:

    Code:
    option confirm:yes-to-all
    criteria timestamp size
    load photos_live photos_backup
    expand all
    select lt.orphans
    copy lt->rt
    select lt.diff.files lt.newer.files
    copyto lt c:\photos_backup\%date%
    To run the script, use the command: bc2 @script.txt
    Chris K Scooter Software

    Comment


    • #3
      Re: Mirror with versioning?

      Thanks Chris for the workarounds...

      In re-reading what I wrote, I should clarify my second bullet point:
      <ul type="square">[*]If a file is new on the left (and doesn't exist on the right), copy it over with the same directory hierarchy
      [*]If a file is newer on the left (and exists on the right), then create a subdirectory on the right under the same hierarchy with today's date and then copy over the changed file to this subdir[/list]

      In other words, let's say I have my setup as:
      Code:
      LEFT SIDE                   RIGHT SIDE
      ---------                   ----------
      E:/                         F:/
        2004/                       2004/
          2004-01-10/                 2004-01-10/
            20040110_1234.jpg           20040110_1234.jpg
            20040110_1236.jpg           20040110_1236.jpg
            20040110_1237.jpg           20040110_1237.jpg
            20040110_1239.jpg           20040110_1239.jpg
          2004-02-14/                 2004-02-14/
            20040214_4232.jpg           20040214_4232.jpg
      Now, let's say it is Jan 7, 2005 and I decide to edit the file "20040110_1237.jpg". I would like the resulting synch (left to right) to end up as:

      Code:
      LEFT SIDE                   RIGHT SIDE
      ---------                   ----------
      E:/                         F:/
        2004/                       2004/
          2004-01-10/                 2004-01-10/
            20040110_1234.jpg           20040110_1234.jpg
            20040110_1236.jpg           20040110_1236.jpg
            20040110_1237.jpg           20040110_1237.jpg
            20040110_1239.jpg           20040110_1239.jpg
                                        edit-2005-01-07/
                                          20040110_1237.jpg
          2004-02-14/                 2004-02-14/
            20040214_4232.jpg           20040214_4232.jpg
      Note how the directory hierarchy is preserved and the "newer file" is simply placed in a dated subdirectory, rather than overwriting the original right-side version.

      Similarly, if I were to edit the same file again on Jan 13, 2005, I'd end up with something like (only showing right side):

      Code:
      F:/
        2004/
          2004-01-10/
            20040110_1234.jpg
            20040110_1236.jpg
            20040110_1237.jpg
            20040110_1239.jpg
            edit-2005-01-07/
              20040110_1237.jpg
            edit-2005-01-13/
              20040110_1237.jpg
          2004-02-14/
            20040214_4232.jpg
      Several backup / mirror / synch products out there try to address this by creating an entire new tree dated at the root of the hierarchy, but this is not very useful, as it doesn't do a good job of preserving the original hierarchy.


      2) Use a BC script.

      It is possible to expand the %date% environment variable in a BC script to create your backup directory. The default for %date% on U.S. English systems in XP is to format the date with / characters, which aren't valid for file names. To change to a different date format, open the Control Panel. Open "Date, Time, Language, and Regional Options". Open "Regional and Language Options". Click Customize in the Regional Options tab. Go to the Date tab. Change the Date Separator under "Short date". I changed mine to the - character.

      And here's the script to use:

      Code:
      option confirm:yes-to-all
      criteria timestamp size
      load photos_live photos_backup
      expand all
      select lt.orphans
      copy lt->rt
      select lt.diff.files lt.newer.files
      copyto lt c:\photos_backup\%date%
      To run the script, use the command: bc2 @script.txt
      Thanks for the script suggestion... This is very similar to what I had original considered doing (apart from your clever idea about changing the OS's date delimiters!), except that the dated directories appear at the root of the resulting tree, rather than at the lowest point in the directory hierarchy.

      From a useability and cleanliness point of view, there is a huge difference. Unfortunately, most backup / mirror / synch programs approach the problem in this sort of way.

      If one could preserve the RHS path and simply append a date-based string, then one would have an extremely robust and intuitive archiving setup!

      Any other ideas or comments?

      Many thanks,
      Cal.

      Comment


      • #4
        Re: Mirror with versioning?

        Hi Calvin!

        That's over all a quite difficult task. I have tried to get your requested results by using a DOS batch-file including a BC-script.
        My results were structures like:

        C:\temp\edit-2005-02-03\2005-01-01.jpg

        or

        C:\temp\edit-2005-02-03\2005-01-01\2005-01-01.jpg

        (the second one is quite close, but the last two folders should change place)

        The main problem is IMHO to determine the folder where the modified file comes from. If this folder (2005-01-01) could be transfered in a variable like %source-dir% it would be no problem.

        But I noticed that it requires a lot of DOS batch programming skills to reach that aim.

        Or is there another script language (like perl, python, cygwin, VB, etc.) available on your pc?
        That might help you further ...


        Bye
        Christoph

        Comment


        • #5
          Re: Mirror with versioning?

          Sorry for misunderstanding your earlier description. I don't see an easy way to do this using the current version of BC.

          I'll add your clarification to the wish list entry for enhanced backup features.
          Chris K Scooter Software

          Comment


          • #6
            Re: Mirror with versioning?

            The "Copy file name" is missing as a script command.

            If it existed, one could be easy process the filenames file with the following DOS batch.

            So, select manually all the files to be backuped, and "copy filenames" thru the context menu; paste the clipboard in a file (via Notepad,...), and execute the following batch file with the created filename as the parameter.
            Code:
            @ECHO OFF
            SETLOCAL ENABLEEXTENSIONS ENABLEDELAYEDEXPANSION
            SET ODD=1
            FOR /F "DELIMS=*" %%F IN (%1) DO (
                SET IS_DIR=%%~aF
                SET IS_DIR=!IS_DIR:~0,1!
                IF "!IS_DIR!" == "-" (
                    IF !ODD!==1 (
                        SET yy=
                        SET NEW_DATE=%%~tF
                        FOR /F "USEBACKQ TOKENS=1-3 DELIMS=/ " %%i IN ('!NEW_DATE!') DO (
                               REM !!! We suppose that the date is in dd/mm/yyy format. !!!
                               REM !!! If not, change the following lines !!!
                               SET dd=%%i
                               SET mm=%%j
                               SET yyyy=%%k
                        )
                        SET MOD_DATE=!yyyy!-!mm!-!dd!
                        SET NEW_FILE=%%F
                        SET ODD=0
                    ) ELSE (
                        SET BAK_PATH=%%~dpF!MOD_DATE!
            echo        MD !BAK_PATH!
            echo        COPY /Y !NEW_FILE! !BAK_PATH!\%%~nxF
                        SET ODD=1
                    )
                )
            )
            To really execute the copy, remove the "echo" in front of the 2 lines near the end.
            If the date is not in the dd/mm/yyyy format, change the script accordingly. You may use the following batch file to test what is returned by your system:
            Code:
            FOR %%F IN (%1) DO ECHO %%~tF
            giving a file name as parameter.
            Michel Dessaintes
            Modérateur du Forum Francophone & Traducteur Officiel de BC2/BC3/BC4 en Français

            Comment


            • #7
              Re: Mirror with versioning?

              Chris, chrroe, Michel --

              Thank you so much for all of the ideas. [I thought I had posted a reply ages ago but perhaps I didn't hit submit?] Anyways, yes, the DOS script looks like it would accomplish part of the task, but would unfortunately not be automated (due to the filename generation). Native support for the folder path date concatenation would be a huge plus.

              In the interim I will look forward to any future updates that might enable such a feature.

              Thanks again & keep up the good work!
              Cal.

              Comment


              • #8
                Re: Mirror with versioning?

                The directory is being created as you suggest (with the change in the short date format), but when I try the following, it copies EVERYTHING:

                LOAD "C:\inetpub\wwwroot"
                FILTER *.bak
                EXPAND ALL
                Option confirm:yes-to-all
                SELECT LT.ORPHANS
                COPYTO LT C:\Backup\%date%

                Comment


                • #9
                  Re: Mirror with versioning?

                  I don't see anything immediately wrong with the script.

                  I ran your script against test folders on my system, and it appeared to copy any files with the *.bak extension, but did not copy files with other extensions.

                  I noticed that you're selecting lt.orphans in your script and you're loading only one folder. In that situation, all files are orphans, so the script should select all files that match the filter.

                  When you say the script copies everything, are you only intending to copy orphan files as compared to another folder, or is the script copying all files regardless of file extension?
                  Chris K Scooter Software

                  Comment

                  Working...
                  X