BC3 "Open with" Multiple Instances bug?

[NL80]

I want to perform a single command on multiple files. I have no problems doing this on BC2. On BC3 it fails to work. When I select multiple instances box and I select for example, 4 files the 4 files are executed four times rather than executing the four as a pair.

Does multiple instances work in BC3?

Below I have provided and example describing the issue. The command line that I am using is to draw a merge arrow between two files. When I enable the multiple instances the command fails.


Draw Merge Arrow

cleartool merge -ndata -to "%f2" "%f1"

The above command works without multiple instances enabled on two files only, but fails when multiple instances box is selected. The feature does not work when applying commands on multiple files.

[Chris]

NL80,

Thanks for reporting the problem. I repeated it on my own system and added it to our bug list to be fixed.

[Giha]

almost 8 years later, this bug seems to be still unfixed in the current version 3.3.13 of BC3 although the help on this topic says:

When the Multiple instances option is enabled, you can select multiple files or file pairs, and execute the operation on all of them once per file or pair of lined up files. With this option, Beyond Compare checks to see if a second file would be given on the command line (eg. "%x2"). If it is, the application will execute on pairs at a time, otherwise it will break the pair up and execute once for each selected item in the pair.

Is it still on the bug list to be fixed?

BC is great software and usually very well documented. So it should be a priority to make it work according to the documented behavior.

Maybe, it it just how BC parses the given command line to check for the presence of the second file? If it does work for a particular command line that was used at the time of programming/testing/documenting, please post it as a sample, ideally with an indication what exactly BC recognizes as a second file.

[Aaron]

Hello,

This is still an open issue in our tracker. BC2 behaved this way, but BC3 and BC4 assume Left and Right (even if selecting only files on the left). We would not backport this change to BC3, but it's still something we would consider for a future patch.

[Giha]

Thanks, Aaron for the quick reply.
I'm actually searching for a workaround for another long term issue that setting to "ignore folder structure" doesn't allow to replicate the subfolder location i.e. "Keep relative folder structure" on one of the sides when copying/moving to the other side. My batch file solution is now working for a single pair of files, but this bug prevents me from running it on a selected list of file pairs. Are you implying that this might work by going back to BC2?

[Aaron]

Re-testing this, I'm actually not seeing the change in behavior I expect (BC2 and BC3/4 are functioning the same). It may be limited to a specific selection and scenario, rather than more broadly. You can install BC2 and test with your specific script, without removing or altering BC4. BC2 and BC4 can be installed on the same system at the same time:
http://www.scootersoftware.com/download.php?zz=dl2_en

If you find this works for you, please email us at [email protected] with a link back to this forum thread for our reference and we can get you a BC2 key to use.

[Giha]

Aaron, thanks for your offer. I still have my old BC2 key. Meanwhile, I found out that BC2 did not yet offer to "ignore folder structure". So, I would need BC3 to produce the list of files to work on.

My scenario is as follows (and I think it is a pretty common one):
My smartphone/camera stores picture/video files on the micoSD card underneath the standard DCIM folder. After syncing with my laptop, theses files are copied into subfolders named according to the date taken. From there, I sort them manually into yet another subfolder structure which is more descriptive of the pictures/videos content and merges them with pictures/videos from several other cameras/sources. The filenames remain the same during all this, so that I can easily line up the sorted copies on the laptop with the original files in the flat folder on the micoSD card by using "ignore folder structure" in BC3.

My aim is now to move the original files on the micoSD into the same subfolder structure, in order to make this context information also available on the smartphone and clean up the multi-gigabyte flat file folder underneath DCIM to receive new pictures/videos with less delays due to saving into a huge folder. I am trying to avoid recopying the files back onto the micro SD card, because this would shorten the lifetime of the flash memory and I would have to chose the relevant pictures from the sorted structure based the EXIF camera information, adding yet another complication, on which BC would choke. For videos, I wouldn't even know how to identify the original source in the file's meta-information.

As far as I know, this is a very common scenario that BC could easily handle if the "ignore folder structure" setting wasn't so limited or if the "open with" would work with multiple instances of file pairs according to the behavior documented in the manual.

I now got as far as extracting a list of the relevant file pairs with the "copy filename" function. Pasted into a text file, this produces a list of files with their absolute paths, odd lines listing the left panel, even lines listing the right panel. A batch script can then realign the odd and even files, filter out the new relative path information and move the files accordingly on the microSD by calling robocopy (in windows) with the appropriate parameters. But this realignment of odd and even files and extracting the relevant relative paths is a very error-prone process, which requires me to reprogram and retest the batch script each time.

BC possesses great alignment functions with regular expressions with could do all this very easily if the above mentioned bugs were fixed. I am surely not the only one waiting for this to happen for almost eight years. Why is this so low on your priorities list? Anybody, who is rearranging a big folder structure and wants to replicate this elsewhere without recopying and deleting many gigabytes of data would profit from this function. As ever more data is stored on flash memory (SSD, memory cards and sticks) it is also becoming increasingly important to keep the amount of data written low in order to extend the lifetime of the data storage and to increase long term data reliability. If BC's own move/rename function cannot do, it needs to allow the implementation via the "open with" function using file pairs in multiple instances, just like its manual says that it does.

With this long post, I hope to raise awareness for the importance of this unresolved issue inside BC / Scooter Software .

[Aaron]

Hello,

Thanks for the example case.

One note, our Open With does currently work with file pairs and multiple instances. The original issue is related to a single side selection (which BC2 could re-interpret and open as pairs).

When selecting the pairs of files, are you select both sides of the comparison so both left and right files are highlighted? You can use the center column to highlight both, or the Shift+Arrow keys to slide the selection from one side, to both, to the other.

If you are still having trouble, one quick test is to create a "BC3" Open with that calls to bcompare.exe "%f1" "%f2". Does this open as tabs for each file pair? Are you running the latest 3.3.13 release of BC3 (all 3.x updates are free for 3.x users)? You can also email us your BCSupport.zip from the Help menu -> Support; Export and we can look over your settings and Open With definition. If you could also send in a full screen screenshot showing your comparison and selection that you execute on (and any helper files that might go with the Open With to investigate) we can better re-create the scenario you are seeing.

[Giha]

Aaron, thanks for your reply.
I am running the latest 3.3.13 release of BC3.

The quick test that you suggested, results in BC opening 4 tabs each comparing one jpg against nothing instead of 2 tabs each comparing 2 jpg against each other. i.e BC3 uses each file pair to call

bcompare.exe "%f1"
bcompare.exe "%f2"

I cannot determine the order in which the 4 calls are made. Each time the 4 tabs open in a different, seemingly erratic order and setting "Wait for previous instance to finish" has no effect on this.



When I switch off "Multiple instances" in then "Open With" options for «bcompare.exe "%f1" "%f2"», then I can only access it in the right click-Open With sub-menu, when a single file pairs is marked and it works as expected: opening the Quick Compare window with the result binary same and offering me a Picture Compare by clicking Open View.

One note, our Open With does currently work with file pairs and multiple instances. The original issue is related to a single side selection (which BC2 could re-interpret and open as pairs).

Could you please post a scenario in which it does work as you say?

If you could also send in a full screen screenshot showing your comparison and selection that you execute on (and any helper files that might go with the Open With to investigate) we can better re-create the scenario you are seeing.

Below is an example of the scenario that I described in my last post:

The aim is now to move the selected 8 files on the left side from their current folder
U:\AP\BC\Test\microSD\DCIM\100ANDRO\ to their respective locations matching up with the right side i.e.
U:\AP\BC\Test\microSD\manually\sorted\Files\2017\0 624 Event C\ and
U:\AP\BC\Test\microSD\manually\sorted\Files\2017\0 429 Event B\ .

Ideally that should be a simple operation directly within BC, which has been on the wish list for several years. But it could also be done by calling a move subroutine using Open With (multiple instances!) The standard MOVE command in windows cannot create folders on the fly. Therefore I am trying to use RoboCopy for this.
The call would look as follows for the first file:

RoboCopy "U:\AP\BC\Test\microSD\DCIM\100ANDRO\." "U:\AP\BC\Test\microSD\manually\sorted\Files\2017\0624 Event C\." "DSC_3634.jpg" /MOV

and as follows for all files if BC would behave as expected

RoboCopy "%p1\." "%p1\..\..\%P2\." "%b1%x1" /MOV

Note that the number of \..\..\ above needs to fit the relative path depth of %p1 and would have to be adapted in a different usage case. It would be helpful to define a parameter such as %B1 and %B2 to designate the base folders on the left and right sides (excluding the last \ if the base folder is at the root). Then the above could be more conveniently written as

RoboCopy "%p1\." "%B1\%P2\." "%b1%x1" /MOV

and would be independent of the relative path depth of %p1.

My real scenario is supposed to move approximately 10 000 picture/video files and the number of files from different sources go in the 100 000s adding up to approximately 300GB. Copying and deleting instead of moving is not a realistic option.

[Aaron]

Ahh, I think I see the problem. The case here is, when selecting a blank side, we do not populate that information: there has to be a target selected.

For my example, it should be an Open with with literally:

bcompare.exe "%f1" "%f2"

When selecting:
file1<>file1
file2<>file2
file3<>file3
file4<>file4

The expected result would be 4 tabs, each comparing the file pair (file1 to file1, etc). This forum thread referred to a disjointed selection that could select file1-4 on just the left, then launch comparisons like file1<>file2, file3<>file4 in two tabs. Re-testing this, I'm having trouble getting the BC2 behavior to reproduce (it's mostly behaving the same as BC3/4), but I remember it triggering in specific scenarios.

The issue I think you are looking for is a related, open issue about passing in an "%f2" if a blank selection is made, to help copy programs. I'll add your notes to our wishlist entry on that subject.

There are instances where customers have wanted to insert other copy programs (for performance or other reasons). Could you go into a bit more detail on why our Move command or Move to Folder command does not work in this scenario for you? Is the RoboCopy Move handling the SD Card in a better way?

Update: Ok, re-reading I see where the adaptive subfolder level would be too tedious to manually move if there are too many different subselections to make. If it were possible to generate a filename filter to limit the view to the specific selection you needed for a specific move, this might help then allow for an easier "Select All Files" -> Move/Move To Folder.

[Giha]

Dear Aaron,
the fact that Open With doesn't populate %f2 or at least %p2 with a blank selection on the right side is yet another issue, which I have seen being discussed elsewhere in the forum. My issue is a different one:

I did exactly what you say: My "Open With" command lines is

bcompare.exe "%f1" "%f2"

and also the associated description is bcompare.exe "%f1" "%f2" which you will find in the Open-With submenu in my first sceenshot, highlighted in light blue. Also visible in both of my screenshots, highlighted in light green, are the selections on both sides, no blank selections. However, I still get the behavior of BC3 just using %f1 twice, once for the left file and once for the right file, all while keeping %f2 blank, as soon as Multiple instances is selected.

Re-testing this, I'm having trouble getting the BC2 behavior to reproduce (it's mostly behaving the same as BC3/4), but I remember it triggering in specific scenarios.

I would be very happy to see a specific scenario where BC3 actually does what you explained i.e. opening multiple tabs of file pairs. For me this works for a single tab only, and only if Multiple instances are turned off.

Could you go into a bit more detail on why our Move command or Move to Folder command does not work in this scenario for you? Is the RoboCopy Move handling the SD Card in a better way?

The main reason is that BC3 doesn't allow to replicate the subfolder structure when Ignore folder structure is selected. This is indicated by the "red Ø over a folder" icon just to the left of the ≈ icon in the toolbar of my second screenshot. I am adding the complication in my scenario that I do not want to move or copy the files on the right. I only want to use their subfolder location and replicate it on the left. So I am moving the files from the left to the left, using the subfolder structure on the right. It would be great if BC could do this directly but alas...

One additional advantage of using RoboCopy is that it can copy also file and folder creation dates, an often demanded feature only recently added to BC4 for files, but not yet for folders. After the files are sorted into the correct structure on the SD Card, I call RoboCopy with Open With to fix the time stamps on the newly created manually\sorted\ folder structure. Here, I have no trouble, because I can do this by calling RoboCopy once for the whole subfolder structure from its top folder down and I can do it without Multiple instances.

• The creation dates of pictures typically indicate, when the picture has been taken (EXIF date taken).
• The modification date shows when the picture was last edited (e.g. cropped, color corrected, etc.)
• The creation date of the event folder indicates, when the pictures where first manually sorted into this event.
• The modification date shows when data about this event has last been updated inside this folder.

With the correct RoboCopy parameters, all the above dates stay preserved. Many other move/copy routines including most of the ones inside BC, mess them up. The nice thing about RoboCopy is, that it can repair a lot of the timestamp mess left after most BC operations and it does it even very fast.

If it were possible to generate a filename filter to limit the view to the specific selection you needed for a specific move, this might help then allow for an easier "Select All Files" -> Move/Move To Folder.

My real use case has hundreds of different folders to move to, not just two as in my example, which I simplified in order to fit it into a single screenshot. They can all be easily selected by switching off the orphan display, then marking them all in a single click and shift-click operation. Subsequently, I re-displayed the orphans, in order to indicate in the screenshot that I only want to move the files already on the SD Card, not the ones which might have been added from different sources into the \manually\sorted folder structure on the right side.

The ideal solution for me would be to select all the files inside the DCIM\100ANDRO folder that have an equivalent on the right side and then chose the BC Move-to-Folder command with the Folder Structure option selection "replicate relative folder structure from the other side" ... if such an option existed

Hopefully, this post sheds more light on my scenario.

[Aaron]

Ah, sorry, I accidentally tested with my BC4 install instead of BC3 when double checking %f1 %f2 with Multiple Instances enabled. Double checking for BC3, this wasn't available and something we fixed for BC4.

I'd suggest installing the trial of BC4 (a separate install of BC3 on Windows) and test out your workflow with this Open With.
http://www.scootersoftware.com/download.php
Does this help with your custom Open With and with the Creation Dates for Files (you'd still need Robocopy for folders, as you point out)?

And I see about your scenario. It's almost like a "Touch Folder Structure" command.

[Giha]

Aaron,
yes, my "Touch Folder Structure" command works almost like the BC internal Touch function when selecting "copy timestamps from other side". However, it copies all available time stamps, modification AND creation dates and can also be extended to copy permissions for files AND folders, if necessary.

I use it very frequently, almost after every bigger sync operation with BC, to clean up the times stamps. It would be my dream if BC would just do this by itself. Frankly, I don't think it would be that much work to implement...

And while you're at it, you could add a feature that I haven't seen anywhere else on the market, that is to propagate the OLDEST CREATION DATE. After doing lots copying, editing and syncing, the creation date gets frequently messed up because it it ignored by many programmers, who have never dealt with heaps of historical data. Like in archaeology, a correct creation date is always very helpful to weed out the rubbish from the useful. Isn't that what BC is all about? At least that's what I am using it for on a daily basis. BTW, Thank you for creating this almost irreplaceable jewel.

I will try BC4. is your offer for the BC2 key also valid for 4? After all, my BC3 help file already promises me that I can do it.

[Aaron]

Hello,

If you write in to [email protected] (with a link to this forum thread for our reference), we can provide key for a previous major version if you need it. Since you already own BC3, you are welcome to try BC2. However, as you mention earlier, BC2 does not allow you to Ignore Folder Structure, so it may not meet your needs. On Windows, you can have installs of BC2, BC3, and BC4 at the same time; either as registered or in trial mode for each. The only conflict is the Windows right-click shell extension. I'd recommend disabling it for all versions but one to avoid any overlap or confusion (and you can always disable/enable different versions in each's Tools menu -> Options dialog).

Part of the issue is it is not just developers who ignore Creation date, but Windows. A regular Windows Copy preserves the Modified date, but it is convention to update the Creation date of the new copy. It's on our wishlist to expand our functions for preserving and touching different timestamps, but there's always a good chance another program or the OS would update them.

[Giha]

Aaron,
your are right, that even then developers of operating systems ignore creation dates or do not deal with them properly. If they were also librarians, historians, forensicists or archaeologists, they would.

That is why I suggested that BC does it better.

If BC regularly synchronizes between several copies of the same folder structure, always the older creation date is the one that is closer to reality (assuming correct dates in all involved systems and no corrupt or missing dates that sometimes come out as 1-1-1970 because this is t=0 in Unix, or some other specific time way back in the past or ahead in the future). So by propagating always the oldest creation date, BC would eventually also copy the original creation date from that location, a file was really originally created in. That original "oldest" date will then eventually propagate to all synchronized copies.

Here it might actually make sense to differentiate between files and folders. If the creation dates of folders behave like they currently do with the Windows file explorer, they show, when the folder was created in this specific location. The effect is that a newer folder creation date than the creation dates of files inside that folder indicate that those files had originally been created elsewhere, which is again a very useful piece of information when synchronizing data.

However, not propagating the oldest creation date for folders should be optional depending on how folder dates are being interpreted by the user.

It even makes sense to propagate the oldest modification date after a positive binary or CRC comparison. The "newer" file has probably been "up"-dated after a ftp or email transfer.

It all depends, whether one is interested in the creation and modification dates of the actual content of a file or only in the reference/pointer to it inside the folder/directory. It seems that most developers are only interested in the latter.

This is now really going off topic in respect to the title of this thread. So I suggest to continue the discussion somewhere else like for example in the thread "Created time stamp".