Unison

Unison is a mature, user‑level file‑synchronization engine that reconciles two replicas of a directory tree across heterogeneous hosts. At its core, it implements a bidirectional delta‑propagation protocol that can detect and resolve conflicts without requiring administrative privileges or kernel modifications. The application is written in OCaml, a statically typed functional language, which gives it strong type safety and a compact runtime. The binary is cross‑compiled for major POSIX systems (Linux, macOS, BSD) and Windows, enabling seamless cross‑platform syncs over SSH or raw TCP.

Technical Stack & Architecture

• Language: OCaml (runtime ~15 MB, no external dependencies)
• Network Layer: Custom TCP protocol with optional SSH tunneling; uses OCaml’s Unix module for socket handling.
• File I/O: Pure POSIX calls (stat, open, read, write) wrapped in OCaml’s abstractions; no FUSE or kernel hooks.
• State Management: Each replica maintains a shadow file (.unison/) containing metadata (hashes, timestamps, version vectors). Conflict detection is performed by comparing these shadow files.
• Delta Transfer: Implements a block‑level transfer similar to rsync’s rolling checksum algorithm, enabling efficient transmission of only changed file segments.
• Extensibility: The protocol is documented; third‑party adapters (e.g., for cloud storage or custom transport) can be written in any language that speaks the same binary format.

Core Capabilities

• Bidirectional Sync: Automatically propagates non‑conflicting changes; presents conflicts with a detailed UI for manual resolution.
• Repeat Mode: Uses inotify (Linux), FSEvents (macOS), or ReadDirectoryChangesW (Windows) to trigger syncs on file system events.
• Compression & Bandwidth Control: Optional LZ4 compression and adjustable transfer rate limits to adapt to slow links.
• Selective Sync: Pattern‑based include/exclude rules; supports file masks and ignore files (.unisonignore).
• Security: Authentication via SSH keys or password; all data is encrypted over the network when using SSH.

Deployment & Infrastructure

Unison binaries are self‑contained; no external services or databases are required. For production use, it is common to run Unison as a systemd service on Linux or a scheduled task on Windows, with optional containerization (Docker) for isolated environments. A typical deployment includes:

1. Installing the binary on both hosts.
2. Generating SSH key pairs for mutual authentication.
3. Configuring a unison.prf profile that specifies paths, protocol options, and conflict handling.
4. Running unison -repeat watch to keep replicas in sync continuously.

Scalability is linear with the number of files and network bandwidth; Unison can handle tens of thousands of small files or a few large ones efficiently. Because it runs in user space, adding more replicas simply means starting additional Unison processes with distinct profiles.

Integration & Extensibility

Developers can integrate Unison into larger workflows by invoking the binary from scripts or CI pipelines. The command‑line interface accepts a rich set of flags for fine‑grained control (e.g., -batch, -auto, -ignore). For deeper integration, the open‑source code exposes a C API wrapper that can be linked into custom applications. Moreover, Unison’s shadow files are plain text, allowing external tools to parse and manipulate sync state without touching the core logic.

Webhooks are not natively supported, but developers can wrap Unison in a small service that listens for file‑system events and triggers syncs, effectively creating a webhook‑style architecture.

Developer Experience

The project’s documentation is extensive: the official wiki covers installation, configuration, advanced options, and troubleshooting. The mailing lists (unison-users@ and unison-hackers@) remain active, providing a low‑latency channel for support and feature discussions. While the core team is small, the codebase is well‑maintained, with a clear commit history and rigorous testing. The license (GPLv3) ensures that any derivative work must remain open source, which is attractive for organizations that value transparency.

Use Cases

• Cross‑platform backups: Sync a Windows laptop with a Linux backup server without requiring SMB or NFS.
• Distributed development: Keep source trees in sync across multiple machines for developers who switch between workstations.
• Offline-first applications: Store a local cache that automatically reconciles with a central repository when connectivity is restored.
• Data migration: Transfer large datasets between data centers with minimal bandwidth consumption.

Advantages Over Alternatives

• Bidirectional, conflict‑aware sync unlike rsync’s uni‑directional nature.
• No kernel involvement – runs as a regular user, simplifying deployment on shared hosting or cloud VMs.
• Cross‑platform parity – a single binary works on Windows, macOS, and Linux.
• Fine‑grained control – profiles, include/exclude rules, and repeat modes give developers precise orchestration.
• Open source & GPL – full visibility into the algorithm and freedom to modify for specialized needs.

Unison’s combination of robust, low‑overhead synchronization logic with a developer‑friendly API makes it an ideal choice for teams that require reliable, bidirectional file replication across diverse environments.