GarageHQ

GarageHQ is a lightweight, self‑hosted distributed object storage system that implements the Amazon S3 API. It is designed to run on commodity hardware and scale horizontally across multiple data centers or “zones.” Each object is split into fixed‑size chunks, which are replicated three times across distinct zones to provide strong durability and resilience against network, disk, or even administrator failures. The architecture is deliberately minimal: a single stateless binary with no external dependencies, written in Go (or similar compiled language), that can be deployed on any x86_64 or ARMv7/v8 machine with at least 1 GB RAM and 16 GB disk. This makes GarageHQ ideal for edge deployments, small‑to‑medium cloud clusters, or as a backup target in heterogeneous environments.

Key Features

• S3‑compatible API – full support for PUT/GET/DELETE, bucket management, and multipart uploads, enabling immediate integration with existing SDKs and tools.
• Zone‑based replication – each chunk is stored in three separate zones; the system automatically detects zone membership and performs cross‑zone replication.
• Conflict‑free data handling – leverages CRDT principles to resolve concurrent writes without centralized locking, inspired by Dynamo and Maglev for efficient routing.
• Minimal resource footprint – a single binary, no external services (no database, message queue, or orchestrator required), allowing fast bootstrap on low‑power devices.
• High availability – built to tolerate intermittent network partitions, high latency (up to 200 ms), and disk failures without data loss.

Technical Stack

The system eschews heavyweight databases or external message brokers, relying instead on file‑based storage and lightweight in‑memory metadata. This keeps operational complexity low while still providing the consistency guarantees required for an S3 API.

Core Capabilities & APIs

• RESTful S3 Interface – supports all standard operations: PUT Object, GET Object, DELETE Object, ListBuckets, CreateBucket, DeleteBucket, and bucket policy management.
• Webhooks & Callbacks – configurable HTTP callbacks on object lifecycle events (create, delete, update), useful for CI/CD pipelines or monitoring.
• SDK Compatibility – any language with an S3 SDK (Python boto3, Java AWS SDK, Node.js aws-sdk) can interact with GarageHQ without modification.
• CLI Tool – a companion command‑line client (not covered here) that mirrors aws s3 commands for scripting and automation.
• Health & Metrics – exposes Prometheus metrics (garage_storage_bytes, garage_replication_latency) and a /healthz endpoint for orchestration systems.

Deployment & Infrastructure

GarageHQ is designed to be deployed as a single binary on each node. It supports:

• Docker & OCI Images – pre‑built images are available for quick container deployment, making it straightforward to run in Kubernetes or Docker Swarm clusters.
• Bare Metal – the binary can be installed on any Linux distribution; no rootless mode is required, though it can run with reduced privileges if the storage directories are appropriately permissioned.
• Horizontal Scaling – adding a new zone simply involves launching the binary on additional machines and updating the cluster configuration. The system automatically discovers peers via a gossip protocol.
• Resource Constraints – with 1 GB RAM and 16 GB disk, a single node can serve tens of thousands of small objects; larger workloads are handled by adding more nodes.

Integration & Extensibility

• Plugin Hooks – developers can inject custom logic into the replication pipeline or API request handling via a simple Go interface, allowing for custom authentication backends or data transformation.
• Webhook API – external services can subscribe to object events, enabling integrations with CI/CD pipelines, monitoring dashboards, or archival systems.
• Custom Storage Drivers – while the default storage uses local files, the architecture allows swapping in cloud block‑storage adapters (e.g., S3, GCS) for hybrid deployments.

Developer Experience

• Zero‑Configuration Start – a single garage.yml file can bring up a cluster; the binary auto‑detects peers via UDP multicast or static peer lists.
• Comprehensive Documentation – the README and API reference cover all endpoints, error codes, and configuration options. The project also hosts a public GitHub repo with issue tracking and community discussions.
• Community & Support – active open‑source contributors maintain the codebase; Slack/Discord channels exist for real‑time help. Licensing is permissive (MIT), encouraging commercial use without royalties.
• Logging & Observability – structured JSON logs and Prometheus metrics facilitate debugging in production environments.

Use Cases