This is version 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 . It is not the current version, and thus it cannot be edited.
[Back to current version]   [Restore this version]

Background#

File based replication can be configured between CrushFTP servers. CrushFTP uses a journaled file transfer system to make sure data changed on one server is changed the same way on both servers. Conflict scenarios should be very rare and difficult to achieve. However, we do not recommend the journaled system as the first choice method if you have a shared storage option available between servers. The journaled system should be the secondary choice when its necessary due to geographically distributed servers. The journaling and replication can *only* account for changes being made through CrushFTP. No outside processes can modify files, write to files, rename files, etc. All access needs to go through CrushFTP, or your journaled system will be out of sync requiring manual intervention to try and correct things. Replication requires an enterprise level license, it is not available in normal licenses.

Preferences#

Go to your preferences, Replication tab.

VFS Replication#

(This is not a common scenario and should be used in rare cases.)

The items starting with "VFS" are what applies to this. The secondary server URL is the URL on the other CrushFTP server. So generally it will be "https://server2.com/".

The local server URL is a path locally that is the base location for all replication. This same exact path needs to exist on the opposite server.

The username and password is what will be used when authenticating to the other server to do file transfers. Typically a username like 'replication' is appropriate here. That user needs to exist on both servers, having the same access to the local folder defined in step 2. This user needs to allow all protocols...not that it will use all protocols, but it will cause replication delays if the user is locked down to only HTTP(S) for example.

The ping interval is how often to check if the server is online if there are pending journal items and the server is offline.

The VFS auto play function should be disabled for this. If its enabled, it will slow down user interaction with a server that is online if the opposite server is down. If enabled, on every interaction with a folder or file transfer or rename and so on, it tries to re-play the journal for pending items...you don't want that. Turn it off. If a server is down, and automated process will play the journal at the ping interval checking for when the server comes back online.

User Manager#

Now its time to configure the User Manager.

We need the replication user created, giving them access to the base folder, with full access, and *without* the replication checkbox enabled. That is critical. If its enabled, it would create a circular loop.



Next we need to give a user access to a folder inside that user_homes area...



...and on their folder, enable the replication checkbox.

How it Works#

Changes that john doe makes to their folder are now replicated to the server configured replication server.
The journaled entries are stored in a folder called "multi_journal" in your main CrushFTP folder. The folder structure contains a hash of the VFS item being used as a folder name to group items to the same VFS location. There will always be the one location. In this are time stamped folders for the changes being made. Each time stamped folder has a config.XML file describing the change and parameters needed to replicate that change. If the change is an upload, there is also a single file called "upload" which is the binary dump of the data being uploaded. While an upload is in progress, this file is written to in parallel so that if the transfer breaks to the remote side, we have the whole file in the journal to utilize in replicating it. If the file upload completes successfully, the file is immediately purged from the journal as well.

Troubleshooting#

If the opposite server goes offline, this folder will start to fill up with uploads that are pending to be delivered to the opposite server. If there is a conflict scenario where a replicated action cannot be performed on the opposite server, you will have to manually look through this folder, sorted by date, and decide what to do with the replication item that cannot be processed.