UAZU-Up-

Symirr -- Symmetrical File Tree Mirroring Tool

Intro | Sync operations | Downloads and manual

Introduction

Symirr is a recursive file/directory synchronisation tool for Linux (and potentially other POSIX systems if ported). It is for mirroring changes made to duplicate copies of the same directory tree kept in two or more locations. This is done in a symmetrical fashion, i.e. there is no sense of a "master copy" -- changes can be made on either side (or both sides) and then sync'd to the other. This is ideal when it is necessary to keep duplicate synchronised copies of mail folders, project directories, and so on, on several different machines. It is also useful for creating a 'teleport' directory to quickly transfer files between various machines.

Advantages:

Limitations:

The current code of Symirr has been refactored many many times. I'm happier having a small codebase so that I can make sweeping global changes whenever I find a better approach. To me this is much better than having a huge codebase that I'm afraid to touch for fear of breaking something. This means always looking for the small/simple/efficient/optimal way of getting the job done, rather than going for the comprehensive/extensive-feature-set style, which perhaps explains to some extent why Symirr's design is the way it is.

Sync operations

There are really two distinct sync operations done by Symirr: the first sync and subsequent syncs. (See the IMPLEMENTATION section in the manual for full details of the algorithms.)

The first time Symirr does a sync, it does not know anything about the history of the files it finds there. Ideally the first sync you would do with Symirr would be between the original directory tree and an empty directory (i.e. letting Symirr copy the files across for you), or you would be syncing to an identical copy of the file tree that you have copied across yourself, either using cp -a or tar or some other tool. However, it is possible that you made a duplicate copy of the file tree at some point in the past and you have made changes on both machines and now you want to use Symirr. Since there is no history information in this situation, Symirr cannot detect deleted files, and neither can it detect clashes (files changed on both sides). For this reason, on the first sync it is important to check over the action command-list file carefully to see that Symirr is doing what you expect it to be doing, to avoid any possible problems. In some cases Symirr inserts comments into this file which provide alternatives.

However, after the first sync, Symirr can accurately detect updates and deletions, and recognise clearly when clashes occur. At this point it is safe to allow Symirr to run unattended (e.g. to run automatically from cron). If clashes are detected, i.e. if file changes have been made on both machines, then *.CLASH-######## files are generated which allow the situation to be examined and resolved later on with no loss of information. This is a much more complicated and accurate sync operation (see the manual for the algorithm).

In addition to the default symmetric basic first-time sync (-B), there are also two asymmetric basic first-time syncs, -S 'local-is-slave' and -M 'local-is-master'. These are designed to duplicate exactly the version of the tree on the master, overwriting or deleting anything on the slave that disagrees. Even a newer file on the slave will be overwritten with an older version from the master, although a warning is given in this case. These are intended for the first-time sync when it is certain that no useful changes have been made to the slave. One example use is to sync an old copy of an MH-style mail tree with the current version, where there may be many deletions required to the old copy which would result in (incorrect) attempts to copy files the other way if the default -B were used. So, -S and -M provide an alternative way to start Symirr off, better in some situations. After this point the option should be dropped, allowing the extended sync mode to take care of all deletions and updates.

Downloads

Documentation

  Manual in HTML format
  CHANGES file
  TODO file

Source-code download

Note that Symirr is still in beta, really: there are still a few bugs in Symirr still to be tracked down and a few known problems still to be fixed, which are described in the TODO file above. However, Symirr already works well and is in daily use by the author. My preference would be to wait until Symirr reaches "absolute perfection" before doing a release, but it is probably best to follow the open-source principle of "release early, release often" and get more people using it. So, if anyone finds a bug, please report it, if possible with a short bash-script which demonstrates the failure. Well-defined repeatable failures help me a great deal in tracking down the bugs. Also, in this way I can add more tests to the test-suite, to further refine and improve Symirr.

  symirr-2.0.9.tgz (127K)
  symirr-2.0.8.tgz (127K)


UAZU-Up- These pages and files, including applets and artwork, are Copyright (c) 1997-2016 Jim Peters unless otherwise stated. Please contact me if you'd like to use anything not explicitly released, or if you have something interesting to discuss.