Simple Folder Copy from Left to Right

[djMot]

I am an administrator for a very popular game server. Weekly, I must change the "maps" (a battle field, of sorts) in rotation for the server. Each map is in a folder. In BC4, I have a Folder Compare session that loads two different FTP servers. On the right is the Master server which contains the master copies of all maps. On the left is the Game server which always has a subset of the folders found on the Master server.

Presently on a weekly basis, I go through and select all the maps folders on the Master(right) server that need to be copied to the Game(left) server and use the Copy button on the toolbar to perform the copy. This involves manually selecting approximately 50 folders on the right to copy to the left. Once the current week's maps have been loaded, the previous week's maps must be deleted from the Game server. That usually happens a day later.

I have tools that let me generate lists of maps (which have the same name as the folders) to run for each new week, and would like to use these lists to build BC4 scripts to automate this copy process.

I'm sorry to have to ask, but I've spent quite a bit of time reading the manual, and just a simple, distinct folder copy from one side to the other is not given as an example. Neither have I seen any request here in the fourm to do this exact thing.

Can you please give me the scripting commands to accomplish these tasks?:

1) With the session I have loaded, copy a folder from the left to the right, regardless of whether it already exists. I don't care if an existing folder gets overwritten with identical content.

2) Delete a folder from the left without involving the right in any way. Folders on the right are sacred and must never be altered or deleted.

Thanks.

[Aaron]

Hello,

First, in your saved Folder Compare's Session menu -> Session Settings, Specs tab, you should enable "Disable Editing" for the right/Master copy. This will protect it from any accidental commands as long as you use this saved session.

1) From a selection, the Copy To Side command on the toolbar or in the right click (on selection) menu should perform as you describe. It will take the selection and copy it to the other side (like a regular Windows Copy or FTP transfer command). If it helps, you can use a combination of Display Filters and selection commands to quickly select "everything" you need to copy, instead of clicking each item.

2) Selecting only the Left side items, right clicking, and issuing a delete command will show the Delete dialog (which also has a checkbox for Left or Right if both sides were selected). Review the dialog before committing the delete and it will delete only the selected side. Enabling "disable editing" from earlier will also help avoid any accidental deletes here, but always review the preview/dialog information before committing. *UNDO* is not available in BC4, so any Deletes (or other file actions like Copy) committed are *permanent*.

[djMot]

Hello Aaron,

My apologies. I was apparently very unclear on what I was asking.

I understand how to perform these operations in the BF4 program itself. And no, sadly you don't have filter tools that would allow me to select completely unique folder names from the master list of folders on the right. That would be akin to reading my mind, lol.

What I was asking for are the scripting commands to perform these copies, and to perform these deletes.

I can write a program to output a BC4 script file (probably using VBA in MS Access), but I have not been able to determine the proper BC4 scripting language syntax. That's what I'm hoping to learn.

[Aaron]

Ah, thanks. I missed which sub-forum the thread was originally posted in.

Scripting commands are a series of sequential commands that mirror actions taken in the interface. As such, you can
load "session name"
expand all
select right.files
copy right->left

Similar actions can be taken to select on the left and delete, but what determines which files on the left should be deleted? If you copied files first, what is considered for a delete when you are performing these steps manually in the interface?

Instead of copies and deletes, would a Sync Mirror of the structure work to make a clone of the Right to the Left (copy anything different, delete anything that doesn't exist in the master side) work? If a full copy of the master isn't needed, is there a filter that can limit the view to only the subset that needs to be acted upon? The script for a sync mirror could then work if this sync logic works for your workflow, which you can test first in the graphical interface.

Please note that all sync actions are similar to copy/delete, and do not support Undo. I would suggest reviewing the preview in the graphical interface with a pair of test folders first, while learning how the sync logic interacts with your set of files. This can be done from a Folder Compare using the Actions menu -> Synchronize -> Mirror Right to Left (using the saved session that has Disabled Editing on your Master side).

[Aaron]

I should add that full syntax and examples for BC Scripting are found in the Help file, Using Beyond Compare chapter -> Scripts subchapter. There is also a full list in the Scripting Reference chapter at the bottom of the table of contents.

[djMot]

No, your Sync Mirror idea will not work because:

1) The files on the left (Game server) must remain until the server is reconfigured (told it has a new set of maps to run) and then restarted to engage the new configuration. Deleting folders on the left while the server was still expecting them would hang the server. The copy and delete operations are typically a day apart just to be sure all is running okay with the new map rotation.

2) There are, lets say, 500 maps on the Master server and typically only 50 on the Game server at any given time. At the end of a week, a new 50 maps are copied over and the previous weeks 50 are removed a day later. There is no overlap of any of the new 50 with the old 50. The only way to select the new fifty to copy over is to know what their names are. Similarly, the only way to know which 50 to remove is to know their name, although a date sort in something like filezilla is effective in identifying which folders were added the week before. For scripting purposes, however, it's assumed the name rather than date would be the preferred selection criteria.

The only way this would work is to do it this way (pseudo code here, because the proper syntax is what I originally asked for in this thread):

load "session name"
select folder1 on right
select folder2 on right
...
select folder50 on right
copy all selected right->left

[OR]

load "session name"
select folder1 on right
copy selected to left
select folder2
copy selected to left
...
select folder50
copy selected to left

[OR]

load "session name"
copy right folder1 to left
copy right folder2 to left
...
copy right folder50 to left


[and for the delete script]

load "session name"
select folder1 on left
select folder2 on left
...
select folder50 on left
delete all selected on left

[etc; you get it]

[Aaron]

Hello,

Ok, I think I understand the scenario. Actually, a "date sort" might be just what we need.

If timestamps are updated during the copy to the other server (some servers do not preserve timestamps during a transfer), you could use a Date filter of older/newer than 5 or 6 days to toggle between either set of maps. If the server does preserve timestamps, you could manually Touch the server side, updating them to the time of the transfer, to mark them for the next sync in a week.

In order to automate, the select command must be used to select everything you wish to act on, which acts on criteria (not name). Script can select "left.newer" or "left.all" but not "folder X". If the comparison status cannot be a determining factor, then a filter (Name or Other) must be in place to include/exclude what you need, then select "all" remaining items. The filter could be a full list of 50 items by name, or we can try to find something more general that can effectively filter between the two sets

I'd recommend testing in the graphical interface first, and once a filter is found, it could be translated into the scripted solution. Since the script offers no preview before acting and has no method to undo, it can be difficult to see if the defined filter is working as you expect. By setting up the interface first, we can then determine the configuration needed, and create the script that performs the same steps.

You can set up a Date filter in the Folder Compare's Session menu -> Session Settings, Other Filters tab. Are you able to make a filter that can represent the "new" or "old" map set you need to copy or delete?

[djMot]

...The filter could be a full list of 50 items by name...

Let's just focus on this for a bit, because this is what I needed to hear. What is the syntax for this in your scripting language?

[Chris]

Example copy script:

log verbose c:\copylog.txt
filter "-.\*;.\map1\;.\map2\"
load ftp://user@gameserver/maps/ ftp://user@masterserver/maps/
select all
copy right->left

Example delete script:

log verbose c:\deletelog.txt
filter "-.\*;.\map4\;.\map5\"
load ftp://user@gameserver/maps/ ftp://user@masterserver/maps/
select all
delete left

Filters are separated with semicolons. The -.\* filters out any files in the root of the base folders. The .\ prefix makes a filter an exact match. If you don't use the .\ prefix, then it matches anywhere in the tree. As an example, the filter "-folder1" will filter out folders named folder1 at any depth in the folder structure.

You can also pass command line arguments into a script using the variables %1 through %9. Example script to pass the filter in as a command line argument:

log verbose c:\copylog.txt
filter "%1"
load ftp://user@gameserver/maps/ ftp://user@masterserver/maps/
select all
copy left->right

Call the script with the command line:
bcompare.exe @script.txt "-.\*;.\map1\;.\map2"

Beyond Compare doesn't support undo, so it's a good idea to test the scripts with some dummy folders before you run them on important data.

Also, to test script strings, load the same folders interactively, then paste the semicolon separated list of filters in the Filters toolbar and hit Enter to see what items will be filtered.

The log verbose script command will list the name of every file or folder copied or deleted.

[djMot]

NOW WE'RE TALKIN' !!
Thank you Chris! I will give this a try over the weekend and let you know how it goes.

[Chris]

Note I just edited my previous post to fix the copy direction of the first (copy) example script.

[djMot]

Hello Aaron and Chris,

The results of my testing are very encouraging, but did not go as expected.

Chris, I used your first script example and modified it for my environment giving me the following:

log verbose "C:\Users\me\OneDrive\XI Admin\DM2\copylog.txt"
filter "-.\*;.\mp_010test\;.\mp_012test\;.\mp_013test\"
load "XI Redirect to DM2"
select all
copy right->left

The "XI Redirect to DM2" profile is the one I use to do all the work manually. When I pasted the filter into the toolbar filter box as you suggested to verify the filter, I got exactly the 3 folders contained in the folder list. So far so good.

When I ran the script, the small monitor window appeared, and it seemed to go through the complete pre-scan that happens whenever I load a profile. It even stopped to tell me there was a file on the Game Server that was older than a file on the Master Server (but completely outside of the filter) and asked me if I wanted to replace it. I said No and it continued. Once it started copying files, it became apparent that it was copying files outside of the 3 folders it was given in the filter, so I canceled the operation. Going back into the BC gui, I found it had created a few dozen empty folders on the Game Server corresponding to folders on the Master and had gone back to start filling them. Clean up was not too big of a deal. I have no idea why it didn't just create an empty folder for all missing folders on the Game server, but I assumed it must be batching the operation in some way.

What I don't understand is the "select all" statement in this script. But I took it on faith that since a filter had been specified, that it meant to select just those in the filter list, but clearly it seems to have taken it quite literally to mean select ALL. In the log file at line 1486, I find:

1/6/2019 6:50:01 PM  >> select all
1/6/2019 6:50:01 PM  >> copy right->left

Followed by commands that start at the top of the folder list.

So, is this "select all" part correct? Is there something else that would have selected only what is on the filter list?

[Aaron]

Hello,

The act of loading the session would load the saved sessions filter settings (overriding the previous command). If you are loading a session, you can load it first, then set the filter after the load to apply it to that session:
log verbose "C:\Users\me\OneDrive\XI Admin\DM2\copylog.txt"
load "XI Redirect to DM2"
filter "-.\*;.\mp_010test\;.\mp_012test\;.\mp_013test"
select all
copy right->left

Alternatively, if you set the filters in session and use multiple sessions, you could load the session (and the included filters) without re-setting them, but this assumes a smaller use case where you could have a handful of saved sessions that wouldn't need frequent editing. The above script, with a manual filter command, is more flexible for editing.

Select all is similar to a Ctrl+A command, but it needs the previous filter command to be fixed. Once the filter is working, then the select "all" (non-filtered items) should similarly work.

[djMot]

YES! Success. Looking at the log, I noticed that the "Load comparison:" statement uses the raw ftp connection syntax, even though I specified the profile name. So I used that syntax instead on my Load statement and placed the filter after that. Only the intended folders were copied. I then created the delete script using the same basic script structure with a "delete left" command. Viola! Perfect!

You guys have no idea what an extreme time suck it has been to do this manually, even with a nice gui like BC4 has. This scripting reclaims a major chunk of time. Once I've automated the map rotation set generation, and script file generation, it's going to be as good as a few clicks vs painstaking selecting, checking, and then double checking. Thank you, Thank You, THANK YOU!


And just a special shout-out to you guys for being in Madison, WI. I live in Middleton, work at the University. A few months back, someone showed me an older copy of BC3 a while back and I was completely taken by its capabilities and how it could be used to help out with these gaming servers. I came here to the Scooter Software website to investigate further. When I found you are right here in Madison, I immediately whipped down some coin for a Pro edition just to support a local software company that makes a really cool program. Thanks again for the help you've provided here and a great piece of software!

[Aaron]

Excellent that it's working and fun shout-out! It's nice to have more time to play on your game server than spent managing it.